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INTRODUCTION 
Subject 

Through  cooperative  agreement  DAMD17-95-2-5023,  the  research  team  sought  to  build  and 
evaluate  an  endoscopic  sinus  surgery  simulator. 

Purpose 

The  purpose  of  this  cooperative  agreement  is  to  find  answers  to  the  following: 

•  Can  surgical  simulation  enhance  the  comprehension  and  performance  of  minimally 
invasive  surgical  procedures? 

•  Are  objective  performances  of  simulated  surgeries  useful  in  the  evaluation  of 
surgical  competence? 

•  What  levels  of  visual  complexity  and  tactile  response  are  necessary  to  achieve 
enhanced  training? 

•  Can  the  above  objectives  be  achieved  at  an  acceptable  cost? 

Scope 

This  research  comprises  two  parts.  Technological  advances  are  required  to  create  a  state-of- 
the-art  simulator  for  training  endoscopic  sinus  surgery  procedures.  In  general,  these 
advances  can  be  categorized  into  haptic  technology,  patient  model  development,  and 
simulation  modeling.  Especially  demanding  is  surgical  interaction  between  surgical 
instruments  and  patient  tissues.  There  is  also  the  question  of  the  efficacy  of  augmented 
virtual  reality  devices  in  training  surgical  procedures.  Specifically,  can  an  endoscopic  sinus 
surgery  simulator  improve  surgical  skill? 

Background 

Surgical  simulation  for  physician  training  is  a  recent  development.  Research  thus  far 
includes  simulation  for  a  variety  of  medical  procedures,  including  abdominal  laparoscopy 
(1-4),  limb  surgery  (5-6),  eye  surgery  (7-8),  plastic  surgery  (9),  gastrointestinal  endoscopy  (10- 
13),  and  epidural  anesthesiology  (14). 

Colonoscopy,  flexible  sigmoidoscopy  and  epidural  simulators  have  employed  haptic 
systems.  The  approach  taken  is  highly  dependent  on  the  specific  requirements  of  the 
simulator.  Haptic  technology  is  attracting  considerable  interest  (14-17). 

Visual  cues  must  be  carefully  selected  for  rendering  the  virtual  patient's  anatomy  (18).  There 
are  a  variety  of  approaches  to  synthetic  image  generation  that  affect  the  rendered  image.  The 
mainstream  rendering  technique  is  conventional  computer  graphics.  This  approach  uses 
high-performance  graphics  hardware  to  render  visual  texture  mapped  onto  polygonal 
surfaces.  Other  approaches  render  volumetric  data  (19).  Noar  and  Beer-Gabel  employ 
videodisk  technology  using  images  acquired  during  live  procedures.  This  technique  requires 
many  thousands  of  actual  endoscopic  images  and  produces  superb  realism,  but  the  loss  of 
operator  ability  to  freely  dissect  patient  tissue  makes  this  method  unreasonable  for  sinus 
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surgery.  Noar  is  attempting  to  combine  videodisk  technology  with  computer  graphics 
imaging  (20).  Other  approaches  to  visual  simulation  are  possible  (11, 21). 

Modeling  of  tissue  deformation  to  simulate  the  interaction  between  a  surgical  instrument 
and  the  patient  model  is  in  itself  a  burgeoning  field.  Cover  provides  a  review  of  the  state  of 
the  art  in  mathematical  deformation  models  (1). 

High-fidelity  simulators  have  a  proven  record  of  increasing  the  readiness  of  military  and 
commercial  flight  crews.  They  hold  similar  promise  for  the  medical  field.  While  computer 
simulation  offers  the  potential  for  standardized  training  and  skill  assessment  without  direct 
patient  involvement  (22),  the  medical  simulation  industry  has  published  no  data  supporting 
that  claim.  This  project  is  unique  in  attempting  to  measure  training  transfer  from  a  high- 
fidelity,  procedure-specific  simulator. 

Readiness  is  the  very  reason  for  the  existence  of  military  medicine.  The  General  Accounting 
Office  (GAO)  dtes  that  in  support  of  the  Persian  Gulf  War,  medical  personnel  .  .  were  not 
qualified  in  the  specialties  to  which  they  were  assigned,  were  not  physically  able  to  perform 
their  jobs,  did  not  have  proper  medical  credentialing  documents,  or  had  not  completed 
training."  (23)  The  Department  of  Defense  (DoD),  in  response  to  these  deficiencies,  enacted 
a  new  policy  requiring  medical  readiness  (24).  The  DoD  directed  the  armed  services  to 
utilize  readiness  training  programs  that  include  "realistic  and  skills  training  and  maximize 
the  use  of  emerging  technology  such  as  distance  learning,  computer  simulation,  and  virtual 
reality"  (24).  The  efficacy  of  such  devices  in  medical  training  is  a  fundamental  issue  that 
requires  systematic  analysis. 

In  summary,  the  emerging  field  of  virtual  reality  and  medical  simulation  brings  promising 
new  embryonic  technology  that,  if  properly  applied,  may  lead  to  enhanced  medical  training 
and  readiness.  To  date,  there  has  been  no  measurement  of  the  training  effectiveness  of  these 
devices.  This  cooperative  agreement  applies  the  latest  technology  toward  the  goal  of 
developing  and  measuring  the  effectiveness  of  an  endoscopic  sinus  surgery  simulator. 
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BODY 

Experimental  Methods  and  Approach 


The  research  team  comprised  the  following  individuals  and  institutions,  each  with  a  unique 
purpose  as  given  below: 


TEAM  MEMBER 

RESPONSIBILITIES 

Lockheed  Martin 

Lockheed  Martin,  prime  contractor  for  the  cooperative  agreement: 

•  Performed  the  simulator  systems  engineering  task 

•  Defined  the  software  architecture 

•  Developed  software  for  rendering,  contact  detection,  dissection,  tissue 
deformation,  CT  display,  student  evaluation,  and  computer-aided 
instruction 

•  Developed  the  instrument  models 

Lockheed  Martin  integrated  the  major  components,  including  the 
simulation  computer,  its  software,  die  tactile  feedback  subsystem  and 
patient  models. 

The  Madigan  Army  Medical 
Center  (MAMC) 

MAMC  provided  a  subject  matter  expert  (SME)  in  the  field  of  sinus 
surgery.  Lieutenant  Colonel  Charles  Edmond  MD  was  responsible  for 
overseeing  all  phases  of  requirements  analysis,  design,  development, 
implementation  and  evaluation. 

Immersion  Corporation 

Immersion  Corporation  developed  the  tactile  feedback  prototype. 

Ohio  Supercomputer  Center 
(OSC) 

OSC  developed  the  standard  virtual  patient  model  and  a  battlefield  trauma 
case. 

Human  Interface  Technology 
Laboratory  (HIT  Lab)  at  the 
University  of  Washington 

Working  with  MAMC,  the  HIT  Lab  conducted  the  evaluation  of  endoscopic 
image  synthesis  fidelity  and  completed  evaluations,  human  factors  activity, 
and  the  "summative”  evaluations  at  project  completion.  Because  the  HIT 

Lab  provided  no  material  component  to  the  simulator  itself,  it  could 
evaluate  the  simulator  without  bias. 

Mr.  Robert  Johnston 

Mr.  Johnston  consulted  on  the  establishment  of  design  parameters  for  the 
simulator. 

Mr.  Robert  Eisler 

Mr.  Eisler  consulted  on  modeling  ballistic  injury. 

In  this  report,  "development  team"  refers  to  those  institutions  that  focused  primarily  on 
simulator  development,  including  Lockheed  Martin,  the  Immersion  Corporation,  OSC,  and 
Robert  Eisler.  The  phrase  "evaluation  team"  includes  those  individuals  and  institutions  that 
focused  primarily  on  the  simulator  evaluation,  i.e.,  MAMC,  the  HIT  Lab,  and  Robert 
Johnston. 

During  Phase  I  of  the  two-phase  cooperative  agreement,  the  first  18  months,  the 
development  team  created  a  simulator  to  serve  as  a  platform  for  evaluation  and  further 
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enhancement.  Developmental  items  include  a  model  of  the  human  sinus  anatomy,  a  system 
to  track  surgical  instruments,  and  simulation  software. 

During  Phase  n,  the  last  six  months  of  the  agreement,  the  HIT  Lab  and  MAMC  evaluated  the 
simulator  at  their  respective  locations.  Simulator  development  continued  with  the 
implementation  of  the  evaluation  team's  suggestions. 

Spiral  software  development  techniques  resulted  in  a  working  simulator  early  in  the  project 
and  increased  its  capabilities  incrementally.  The  C++  simulator  software,  developed 
according  to  current  object-oriented  design  principles,  created  a  robust  foundation  for 
iterative  refinement  in  response  to  simulator  evaluation. 

Prior  to  Phase  n,  the  evaluation  team  developed  criteria  to  assess  simulator  functions  with 
respect  to  training  efficacy.  At  the  beginning  of  Phase  n,  after  simulator  deployment,  several 
subjects  used  the  simulator  to  provide  information  about  its  usability  and  success  in  training. 
Refinements  to  the  simulator  reflected  results  of  tins  formative  evaluation  prior  to  formal 
data  acquisition.  After  the  initial  refinements,  evaluators  used  a  frozen  simulator 
configuration  to  acquire  data  from  as  many  subjects  as  were  available. 

Dr.  Charles  Edmond  established  performance  criteria  which  guided  the  incorporation  of 
multi-level  training  scenarios  and  computer-automated  student  assessment  into  the 
simulator.  The  evaluation  team  gathered  data  using  five  standard  techniques: 

•  Observation 

•  Performance  testing 

•  Think-aloud 

•  Audit  trails 

•  Debriefing 

They  observed  the  subjects  to  see  how  well  they  performed  critical  tasks  at  novice, 
intermediate  and  advanced  levels.  The  simulator  computed  scores  based  on  the  established 
performance  criteria.  Subjects  verbalized  their  thoughts  while  performing  simulator  tasks. 
Video  cameras  recorded  each  procedure.  Further,  the  simulator  recorded  the  subjects'  actions 
and  numerical  scores  to  facilitate  a  thorough  analysis  of  student  performance.  Post-trial 
interviews  determined  what  was  difficult,  easy,  successful,  unsuccessful,  preferred,  and 
disliked.  Analysis  of  data  gathered  by  these  procedures  will  guide  further  revision  and 
development. 

Investigators  solicited  subjects  from  three  distinct  groups: 

•  Non-MDs  with  general  intelligence  and  psychomotor  abilities  roughly  comparable 
to  the  average  otolaryngologist 

•  Non-ENT  physicians  from  a  variety  of  specialties 

•  ENTs  with  wide-ranging  ESS  experience 
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Investigators  focused  on  these  three  groups  to  establish  a  baseline  and  asymptote  for  the 
evaluation  of  the  efficacy  of  the  simulator  in  training  otolaryngology  residents.  The  first  two 
groups  also  allowed  a  "shakedown"  of  the  system  and  research  protocols  without  using 
valuable  ENT  resident  subjects,  and  provided  baseline  scores  for  subjects  unfamiliar  with 
ENT  procedures.  Finally,  the  non-ENT  physician  group  could  provide  valuable  input  about 
the  extensibility  of  the  simulator  to  non-ENT  applications. 

Twelve  volunteers  from  the  University  of  Washington  College  of  Engineering  comprised  the 
non-MD  group.  These  subjects  ranged  in  age  from  23  to  54,  and  included  graduate  students, 
professional  staff  and  faculty.  All  had  had  some  experience  with  simulation  and  virtual 
reality.  None  had  previously  used  an  endoscope  or  attended  medical  school. 

Eight  University  of  Washington  MDs  from  specialties  other  than  otolaryngology  provided 
extensive  feedback  on  the  simulator  design  and  utility  for  other  medical  and  surgical  tasks: 
three  video-endoscopic  surgeons,  two  radiologists,  a  neurosurgeon,  a  cardiologist,  and  an 
anesthesiologist.  Four  of  these  subjects  also  provided  us  with  complete  trial  performance 
data. 

Twelve  staff  and  resident  ENTs  (1  female,  11  males)  from  MAMC  served  as  subjects. 
Subjects  in  this  group  ranged  in  age  from  28  to  46,  with  a  mean  age  of  35.2  years,  and  a 
standard  deviation  of  6.15  years.  One  was  left-handed,  10  were  right-handed,  and  the 
handedness  of  one  was  unknown.  ENT  experience  for  this  group  broke  down  as  follows: 

•  Staff  (averaging  five  years  of  training,  six  years  of  practice  and  more  than  100  ESS 
procedures  performed) 

•  R2s  (with  an  average  of  1-5  ESS  procedures  performed  or  observed) 

•  R3  (with  6-20  ESS  procedures  performed  or  observed) 

•  R4s  (with  an  average  of  21-100  ESS  procedures  performed  or  observed) 

•  R5s  (with  an  average  of  21-100  ESS  procedures  performed  or  observed) 

Assumptions 

None. 

Procedures 

Investigators  followed  data  collection  procedures  approved  by  their  respective  Internal 
Review  Boards  (IRBs).  IRBs  at  MAMC  and  The  University  of  Washington  approved 
protocols  for  data  collection. 

To  begin,  investigators  briefed  the  subjects  on  the  procedure.  Each  subject  signed  an  IRB- 
approved  consent  form  and  completed  a  pre-trial  questionnaire,  providing  information 
about  their  background  and  experience  that  might  be  relevant  to  their  performance. 

A  video  camera  recorded  the  procedure  from  that  point  forward.  Investigators  familiarized 
each  subject  with  the  simulator,  providing  techniques  for  handling  the  instruments.  Subjects 
watched  as  the  simulator  demonstrated  the  actions  they  were  to  perform.  Investigators 
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encouraged  subjects  to  think  aloud,  prepared  the  simulator  to  capture  the  subject's  actions, 
and  started  the  simulation.  The  subjects  performed  the  trial  while  the  videotape  and 
simulator  recorded  their  actions.  Upon  completion  of  the  exercise,  the  simulator  stored 
student  actions  and  scores  to  disk  and  the  subjects  completed  post-trial  questionnaires. 
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RESULTS  AND  DISCUSSION 
The  Simulator  -  Technical  Overview 

The  simulator  allows  performance  of  limited  endoscopic  sinus  surgery  on  a  virtual  patient 
using  replicas  of  an  endoscope  and  other  surgical  instruments.  The  student  surgeon  freely 
explores  the  virtual  anatomy  by  manipulating  the  simulated  endoscope.  Using  a  simulated 
needle,  the  student  injects  a  vasoconstrictor  into  sinus  tissue,  causing  it  to  blanch.  The 
student  displaces  the  middle  turbinate  toward  the  septum  with  a  simulated  freer,  gaining 
access  to  deeper  recesses  of  the  sinus  cavity.  The  student  dissects  the  ethmoid  bulla,  middle 
turbinate,  anterior  ethmoid  cells,  uncinate  process,  sphenoid  ostium  and  maxillary  ostium 
with  any  of  a  dozen  dissection  instruments.  All  instruments  except  the  endoscope  provide 
force  feedback  in  three  axes,  allowing  the  student  to  feel  virtual  tissue  through  the 
instrument's  handle. 

The  student  starts  training  with  an  abstract  novice  environment  instead  of  a  virtual  patient. 
Using  this  environment,  the  student  performs  tasks  and  becomes  familiar  with  the  suite  of 
surgical  tools.  This  novice-level  training  improves  hand-eye  coordination  through 
immersive  experience.  When  ready,  the  student  moves  to  intermediate-level  training  where 
abstract  training  aids  appear  in  the  context  of  sinus  anatomy.  Aids  for  navigation  and 
injection  are  identical  to  their  novice-level  counterparts.  They  prompt  the  student  to  perform 
the  appropriate  tasks  while  learning  the  anatomy.  At  the  advanced  level,  the  student 
performs  surgical  procedures  without  the  benefit  of  abstract  computer-based  aids. 


Figure  1.  Endoscopic  Surgical  Simulator  Components 

Figure  1  illustrates  the  principal  components  of  the  endoscopic  sinus  surgery  simulator.  The 
system  has  two  computers  linked  by  an  Ethernet  interface.  A  Silicon  Graphics  Incorporated 
(SGI)  computer  serves  as  the  simulation  host  platform.  The  haptic  system  controller  PC 
performs  high-rate  software  control  over  replicas  of  surgical  instruments.  The  simulator’s 
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endoscope  monitor  is  similar  to  that  used  in  an  operating  room.  A  separate  instructor 
monitor  displays  a  graphical  user  interface  for  control  of  the  simulation.  Speakers  provide 
audible  feedback  to  the  student. 

The  endoscopic  simulator  is  shown  in  operation  in  Figure  2.  The  student  manipulates  the 
endoscope  using  his  right  hand  to  control  its  position  and  orientation.  As  the  endoscope 
moves,  its  monitor  displays  the  anatomy  from  the  vantage  of  the  endoscope  tip.  (The 
endoscope  is  not  visible  in  the  monitor,  just  as  a  video  camera  is  not  visible  in  its  own  video.) 
The  student  holds  the  endoscope  in  a  position  to  view  the  surgery  and  simultaneously,  with 
his  left  hand  in  a  scissors-like  grip,  manipulates  the  surgical  instrument.  He  steadies  the 
virtual  surgical  instrument  within  the  field  of  view  of  the  endoscope  and  operates  on  the 
virtual  anatomy.  The  student  senses  it  by  touch,  and  then  injects,  displaces  or  dissects  it.  He 
"feels"  the  virtual  anatomy  through  forces  applied  by  the  electro-mechanical  haptic  system. 
The  view  in  the  endoscope  monitor  continually  updates  as  he  manipulates  the  endoscope 
and  surgical  instrument. 


Figure  2.  Endoscopic  Surgical  Simulator  in  Action 


8 


DAMD17-95-2-5023 
Final  Report 


The  Simulator  -  Endoscope  Monitor 

The  endoscope  monitor,  shown  in  the  upper  left  corner  of  Figure  2,  displays  images  as  seen 
through  the  virtual  endoscope.  The  20-inch  monitor,  of  the  type  used  in  an  operating  room, 
is  driven  by  the  simulation  computer  with  a  color-composite  480  x  640  signal  identical  to  the 
operating  room  video  format. 

The  Simulator  -  Haptic  System 

The  haptic  system  comprises  a  high-speed  Pentium  computer  and  an  electro-mechanical 
component  (Figure  3).  It  tracks  the  position  and  orientation  of  the  endoscope  through  a 
mechanical  apparatus  external  to  the  mannequin's  head,  and  tracks  another  surgical 
instrument  via  a  second  mechanical  apparatus  inside  the  mannequin's  head.  The  haptic 
system  monitors  the  closure  state  of  the  instrument's  scissors-like  grip.  As  the  user 
manipulates  these  physical  replicas  of  an  endoscope  and  surgical  instrument,  the  connected 
electro-mechanical  hardware  senses  their  position  and  orientation  in  all  six  degrees  of 
freedom.  The  haptic  system  PC  reads  and  transfers  the  complete  state  of  both  tools  to  the 
simulation  computer.  The  haptic  system  applies  force  in  three  axes  to  the  distal  tip  of  the 
surgical  tool,  simulating  haptic  cues  associated  with  surgery. 

Investigators  gave  the  name  "Impulse  Engine  3  Grounded  Motors"  (3GM)  to  the  apparatus 
that  simulates  forces  on  the  surgical  instrument  because  the  apparatus  incorporates  a 
revolutionary  design  to  support  all  the  motors  by  the  ground.  This  approach  gives  the 
mechanism  low  weight  and  inertia.  Investigators  also  optimized  the  device  for  the 
workspace  inside  the  mannequin's  head.  The  3GM  supplies  at  least  4  Newtons  of  force 
display  in  three  dimensions,  low  inertia  and  backlash,  and  six  dimensions  of  location 
sensing.  Sinus  surgery  is  inherently  a  3D  application.  Higher  degrees  of  freedom,  used  to 
display  torques,  are  not  necessary  because  most  forces  encountered  during  surgery  occur  at 
the  tool's  tip. 

To  obtain  3  degrees  of  freedom,  the  device  is  essentially  a  five-bar  mechanism  (which  is  2D) 
with  an  additional  axis  of  rotation.  A  gimbal  that  attaches  the  surgical  instrument  to  the 
endpoint  of  the  3GM  adds  three  degrees  of  rotation.  These  joints  also  use  encoders  and 
capstan  drives  and  are  as  small  as  possible,  limiting  interference  with  the  inside  of  the  model 
head.  They  are  mechanically  robust,  and  provide  angular  swings  of  at  least  30°  in  each 
direction.  An  analog  Hall  Effect  sensor  with  a  resolution  of  better  than  1°  measures  the  angle 
of  the  scissors-like  grip  on  the  tool.  The  3GM  and  gimbal  have  resolutions  of  0.1  mm  and 
0.3°,  respectively.  A  haptic  display  should  have  low  inertia  in  an  attempt  to  make  the  user 
unaware  of  the  presence  of  the  device.  Force  feedback  cannot  compensate  for  inertia  because 
this  would  require  instantaneous  acceleration  measurements.  A  twisting  cable  design  allows 
grounding  of  all  three  motors,  yielding  a  low  inertia  device.  The  linkages  are  lightweight 
carbon  fiber  and  aluminum  parts.  Zero  backlash  is  another  requirement  for  a  "transparent" 
device.  A  cable  drive  system  makes  a  very  smooth  transmission,  with  virtually  no  friction 
and  backlash.  Encoder  readings  at  each  joint  and  forward  kinematics  calculate  the  position  of 
the  endpoint  of  the  surgical  instrument  in  real  time. 
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Figure  3.  Electro-Mechanical  Haptic  System  Component 

The  haptic  system  does  not  apply  force  to  the  replica  of  the  endoscope.  The  endoscope 
replica  connects  to  a  commercially  available  microscribe  from  Immersion  Corporation. 
Investigators  modified  the  microscribe,placing  an  endoscope  replica  at  its  tip  and  adding  a 
roll  sensor. 
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The  200-MHz  Pentium  PC  dedicated  to  the  haptic  system  provides  high  bandwidth  force 
updates  at  rates  exceeding  3000  Hz.  This  minimizes  latencies  in  force  calculations,  and 
results  in  a  stable  force  that  is  stiffer  than  would  be  possible  at  lower  iteration  rates.  It 
communicates  the  surgical  movements  to  the  simulation  computer  sixty  times  per  second.  At 
simulation  computer  display  rates,  the  haptic  system  receives  a  simplified  patient  model  in 
the  neighborhood  of  the  tip  of  the  surgical  instrument.  That  neighborhood  model  becomes 
the  patient  for  the  purpose  of  force  calculations.  Using  this  approach,  the  student  can  freely 
explore  and  sense  by  touch  the  entire  patient  anatomy. 

The  Simulator  -  Haptic  System  Software 

The  patient  model  comprises  over  25,000  surfaces.  Manipulation  of  the  sinus  tissue  through 
dissection,  injection,  and  deformation  applies  to  more  than  50%  of  the  modeled  sinus 
surface.  The  haptic  system  must  interact  with  these  surfaces  at  extremely  high  iteration  rates 
to  provide  even  moderate  stiffness.  To  aid  the  haptic  computer  in  this  task,  the  simulation 
computer  simplifies  the  patient  model  to  a  box  in  the  neighborhood  of  the  surgical  tool's  tip. 
Every  frame,  15  to  60  Hz,  the  simulation  computer  senses  the  location  of  the  patient  model 
from  the  vantage  point  of  the  tool  tip  by  "looking"  to  the  left,  right,  up,  down,  and  straight 
ahead.  It  creates  one  plane  that  is  perpendicular  to  each  of  the  five  sensing  line  segments 
and  sends  the  resultant  five-plane  model  to  the  haptic  computer.  The  haptic  computer 
receives  the  updated  five-plane  model  and  uses  it  for  surgical  tool  interaction  until  another 
model  arrives  in  17  to  67  milliseconds  (ms).  Based  on  this  model,  the  haptic  computer 
controls  the  forces  applied  by  the  motors  at  rates  exceeding  3000  Hz.  With  this  approach,  the 
student  can  feel  any  part  of  the  simulated  anatomy  without  restriction. 

The  system  simulates  a  range  of  stiffness  based  upon  the  CT  density.  Very  bright  or  dense 
CT  objects  are  stiffer  than  dark  objects.  Ultimately,  Dr.  Edmond  drew  from  his  clinical 
experience,  choosing  to  minimize  the  range  of  stiffness  between  hard  and  soft  tissue.  Even 
the  maximum  stiffness  obtainable  through  the  haptic  system  felt  like  mid-range  sinus  tissue. 

Sinus  tissues  vary  in  stiffness  from  soft  to  bony  structures.  Surgeons  palpate  these  with 
surgical  instruments  to  aid  in  differentiating  among  sinus  structures.  While  the 
development  team  considered  this  to  be  important  tactile  feedback,  the  haptic  device  did  not 
supply  enough  stiffness  to  simulate  the  feel  of  dense  bone,  and  attempts  to  simulate  it 
resulted  in  vibration  and  instability. 

As  the  cooperative  agreement  progressed,  the  importance  of  palpation  cues  diminished  in 
relation  to  the  importance  of  tool-specific  haptic  responses.  For  example,  a  needle  that 
penetrates  tissue  should  "stick,"  reassuring  the  student  that  injection  can  occur.  Haptic 
forces  then  hold  the  needle's  tip  in  place.  Without  this  haptic  support,  some  students  simply 
could  not  inject  targets  in  the  novice  model. 

As  another  example,  the  distal  tip  of  a  forceps  grabs  tissue  as  it  dissects,  holding  itself  steady 
during  the  dissection.  Without  this  tool-specific  response,  students  had  difficulty  steadying 
the  tip  during  dissection.  The  simulator  also  mimics  the  vibration  of  the  microdebrider. 
That  vibration  increases  as  the  tool  encounters  tissue,  reassuring  the  student  of  tool  position 
when  the  only  other  position  information  comes  through  the  monocular  endoscope. 
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Although  the  haptic  device  fell  short  of  its  goal  to  provide  distinguishable  palpation  among  a 
variety  of  tissue  types,  determination  of  tool  location  by  touch  and  tool-specific  haptic 
responses  justifies  its  expense. 

The  Simulator  -  Simulation  Computer 

The  simulation  computer  is  the  heart  of  the  endoscopic  sinus  surgery  simulator.  It 
incorporates  an  Onyx  computer  with  four  R4400  CPUs  and  Reality  Engine  II  graphics.  The 
real-time  simulation  software  senses  the  computer  configuration,  dynamically  configuring 
itself  to  take  full  advantage  of  its  hardware.  For  example,  upon  sensing  four  CPUs,  the  real¬ 
time  simulation  software  spawns  four  parallel  processes,  one  for  each  CPU.  Simulator 
latency  differs  widely,  depending  on  number  of  CPUs  and  frame  rate.  Frame  rate  in  turn 
depends  upon  number  of  CPUs,  system  architecture,  graphics  speed,  and  CPU  speed. 

The  Simulator  -  Instructor  Monitor 

The  21-inch  instructor  monitor  is  an  integral  part  of  the  simulation  computer.  When  the 
simulator  is  off-line,  the  monitor  serves  as  the  display  for  software  development  and  UNIX 
administration  tasks.  When  the  simulator  is  active,  simulation  software  takes  over  the 
display  and  presents  both  the  endoscope  view  and  the  graphical  simulation  control  interface. 

The  instructor  interface  provides  control  of  the  simulation  state,  instrument  selection  and 
patient  selection.  It  runs  as  a  separate  UNIX  X-window  application  using  Motif  Widgets. 
The  instructor  display  is  depicted  in  Figure  4.  The  circular  window  (left)  is  the  endoscope 
camera  view,  which  is  created  under  control  of  the  real-time  simulation  process.  This 
window  appears  in  the  instructor  monitor  as  shown  in  the  figure,  but  also  comprises  the 
entire  view  at  the  endoscope  monitor,  as  it  would  in  the  operating  room.  The  interface 
permits  the  user  to  select  an  endoscope  with  standard  offset  angles,  and  choose  one  of  15 
surgical  instruments.  It  displays  and  provides  control  of  the  simulator  state.  A  pop-up 
configuration  window  allows  selection  of  a  patient  model,  student,  proctor,  or  recorded 
procedure  for  playback. 

In  current  medical  practice,  surgeons  routinely  scan  patients  prior  to  surgery  to  obtain 
computer  tomography  (CT)  images.  They  study  the  images  before  and  during  the  surgery. 
Using  a  recent  innovation  called  image-guided  surgery,  technicians  spatially  register 
surgical  instruments  to  the  patient.  The  image-guided  surgery  device  fetches  digitized  CT 
images  in  the  plane  of  the  instrument  and  displays  them  on  a  workstation  to  aid  the 
surgeon's  spatial  awareness.  The  simulator  implements  this  feature  in  the  following  manner. 
The  haptic  system  tracks  the  surgical  instruments  in  the  patient  coordinates.  The  graphical 
user  interface  retrieves  the  coronal  CT  image  of  the  plane  nearest  to  the  tip  of  the  selected 
surgical  tool.  A  cursor  marks  the  tool's  position  within  the  CT  image.  This  feature  is  similar 
to  commercially  available  image-guided  surgical  devices.  Figure  4  shows  the  CT  slice  on  the 
right  side  of  the  display. 
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The  Simulator  -  Speakers 

Stereo  speakers  and  a  sound  card  supply  audible  feedback  to  the  student,  in  the  form  of  the 
sounds  of  a  heart  monitor  and  recorded  voice  messages.  The  messages  coach  and  report 
progress  to  the  student. 

The  Simulator  -  Heart  Monitor 

The  heart  monitor  portion  of  the  simulation  generates  audible  beeps  similar  to  operating 
room  equipment.  Upon  injection  of  virtual  patient  anatomy,  the  heart  rate  increases  rapidly 
and  gradually  returns  to  normal.  Instructors  may  force  an  abnormally  high  rate  to  test 
student  awareness. 


Figure  4.  Instructor  Monitor  -  User  Interface 


The  Simulator  -  Rendering  Approach 


Rendering  the  patient  and  tool  models  came  down  to  two  alternatives,  the  volumetric 
approach  and  polygonal  rendering.  To  decide,  one  must  consider  image  quality,  rendering 
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speed,  and  ease  of  data  manipulation  to  simulate  surgical  interaction.  Voxel-based  image 
quality  degrades  at  close  view  and  rendering  speed  can  be  slow  (25).  Polygonal  rendering 
delivers  superior  quality  images  at  higher  display  rates.  Recognition  of  and  navigation 
through  the  anatomy  is  critical  in  sinus  surgery,  and  cannot  be  adequately  simulated  using 
volume  rendering.  An  advantage  of  volumetric  data  sets  is  that  dissection  mathematically 
reduces  the  removal  of  voxels  through  table  look-up.  There  may  be  no  advantage  for  tissue 
deformation  with  either  approach.  Early  in  the  program,  the  development  team  spoke  with 
computer  graphics  expert  Jeff  Roberts,  manager  of  SGI's  Visualization  Technical  Center  in 
Farmington,  MI.  He  offered  advice  on  current  rendering  rates  and  capabilities  of  SGI 
volume  rendering  software  versus  polygon  rendering  software.  At  the  conclusion  of  that 
meeting,  the  development  team  decided  to  use  polygons. 

Volumetric  rendering  may  become  appropriate  for  surgical  simulation  in  the  near  future. 
SGI  specifies  both  volumetric  and  polygonal  capacity  for  their  products.  This  would  indicate 
that  the  computer  industry  is  starting  to  recognize  the  importance  of  volume-based 
rendering.  Currently,  however,  volume  rendering  does  not  adequately  represent  the  entire 
sinus  cavity  in  real  time,  without  loss  of  detail. 

The  simulator  uses  Performer,™  an  SGI  software  layer  built  upon  the  industry  standard 
Open  GL.  Immaturity,  lack  of  speed  and  fuzzy  images  disqualified  volume-based  rendering 
techniques. 

Investigators  attempted  to  simulate  the  endoscope  image  as  closely  as  possible  to  its 
appearance  during  an  actual  procedure.  The  image  fidelity  demands  of  the  endoscopic 
surgery  are  significant  and  differ  sharply  with  flight  simulation  and  conventional  virtual 
reality  applications.  Unlike  simulating  terrain  on  a  flight  simulator,  the  close  proximity  of 
the  anatomical  structures  causes  a  high  percentage  of  the  polygons  that  represent  the  sinuses 
to  fall  simultaneously  within  the  viewing  frustum.  This  places  a  heavy  load  on  the 
simulation  computer  and  graphics  hardware.  All  illumination  originates  from  the 
endoscope,  creating  an  image  with  a  high  degree  of  depth  attenuation.  Fortunately, 
shadowing  is  unnecessary  because  of  the  co-linearity  between  the  imaging  optics  and  the 
light  source.  The  anatomy  is  moist  and  reflective,  creating  numerous  conspicuous  specular 
highlights  which  yield  important  depth  cues.  Investigators  applied  high-quality  visual 
texture  to  title  geometry  of  the  virtual  patient,  generated  from  the  surface  reconstruction  of 
the  Visible  Human  Project  data. 

Rendering  the  complex  sinus  anatomy  requires  thousands  of  shaded  texture-mapped 
polygons.  This  results  in  poor  real-time  performance  even  for  expensive  state-of-the-art 
graphics  hardware.  To  address  this  concern,  the  development  team  optimized  the  rendering 
algorithm  by  limiting  the  yon  plane  of  the  viewing  frustum  to  a  distance  of  25  mm,  which  is 
the  minimum  distance  Dr.  Edmond  accepted.  Rendering  software  culls  out  structures  that 
are  outside  the  viewing  frustum.  This  yielded  a  substantial  improvement  in  rendering 
speed. 

Another  gain  in  performance  resulted  from  hierarchical  organization  of  patient  model 
polygons.  Simulation  software  collects  polygons  that  are  close  to  one  another  in  space, 
enclosing  them  in  a  sphere  for  testing  against  the  viewing  frustum.  Simulation  software 
further  compiles  these  collections  into  higher  level  groups,  forming  a  tree  with  a  sphere  at 
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each  node.  Investigators  found  that  a  tree  with  5  to  8  levels  yields  optimal  performance.  A 
simple  test  of  any  sphere  against  the  viewing  frustum  may  discard  that  node  and  all  of  its 
descendants  from  processing  by  the  rendering  pipeline.  The  same  hierarchy  allows  efficient 
tree  traversal  for  surgical  interaction. 

The  Simulator  -  Latency 

Two  primary  factors  determine  the  quality  of  the  simulation,  latency  and  patient  model 
fidelity.  For  purposes  of  this  discussion,  patient  model  fidelity  equates  to  the  number  of 
polygons  on  the  patient  model.  Latency  is  the  time  from  instrument  movement  to  the  change 
in  its  display  at  the  endoscope  monitor.  Latency  and  fidelity  are  inversely  proportional. 
Flight  simulation  experience  has  led  to  a  demand  of  latency  of  less  than  100  ms.  Early  in  the 
course  of  this  cooperative  agreement,  the  goal  of  a  100-ms  latency  drove  the  simulator 
design. 

In  the  evaluation  phase.  Dr.  Edmond  tested  patient  models  that  varied  the  load  on  the 
simulation  computer.  He  preferred  to  relax  the  latency  requirement  in  favor  of  higher 
fidelity  anatomical  models.  The  tested  system,  deemed  acceptable  by  Dr.  Edmond,  has  a 
latency  in  the  range  of  200  ms  and  a  20-Hz  iteration  rate.  Using  this  device,  the  evaluation 
team  did  not  observe  any  phenomena  such  as  pilot-induced  oscillation  that  is  common  in 
flight  simulators  with  poor  response.  It  is  possible  to  conclude  that  the  slow,  deliberate 
surgical  motion  used  in  sinus  surgery  allows  a  higher  latency  than  may  be  tolerated  in  flight 
simulators. 

The  Simulator  -  Surgical  Interactions 

Flight  simulator  databases  consist  primarily  of  terrain  whose  elevation  is  a  function  of 
latitude  and  longitude.  (The  industry  refers  to  this  as  2.5D.)  The  user  cannot  manipulate  the 
environment.  Surgical  simulation  is  far  more  demanding.  The  viewing  frustum  may  enclose 
numerous  overlapping  hidden  surfaces  that  represent  complex  sinus  cavities.  The  rendering 
pipeline  uses  much  of  its  bandwidth  to  process  them,  even  though  they  contribute  nothing  to 
the  rendered  image.  Tissue  manipulation  also  challenges  the  capabilities  of  simulation 
technology.  The  sinus  cavity  must  be  fully  three-dimensional  (3D),  with  infinite  possibilities 
for  alteration  by  removal  or  manipulation  of  tissue. 

In  the  object-oriented  software  design,  the  surgical  interactions  are  methods  of  the  software 
objects  that  make  up  the  patient  model.  Each  anatomical  object  responds  to  surgical 
manipulation  with  its  own  dissection  and  deformation  methods.  This  facilitates  tailoring 
behavior  for  each  of  the  various  tissues  of  interest.  The  simulator  implements  dissection  in  a 
limited  way  by  causing  tissue  to  vanish  when  dissected,  exposing  bone  fragments  for  the 
student  to  extract. 
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Bleeding  simplifies  to  a  limited  number  of  "bleeders."  Upon  dissection  of  a  bleeder,  a 
textured  blood-red  sphere  begins  to  grow,  its  volume  increasing  uniformly  with  time.  The 
spheres  reduce  in  size  when  in  contact  with  one  of  the  suction  instruments.  Each  sphere 
immediately  degrades  the  visibility  through  the  endoscope  on  contact,  necessitating  the 
wiping  of  the  endoscope  on  a  pad  placed  on  the  forehead  of  the  mannequin.  These 
simplifications  do  not  significantly  compromise  training  and  a  fully  realistic  simulation 
would  be  a  difficult  undertaking.  Figure  5  depicts  the  removal  of  bone  chips  in  dissection 
and  the  growth  of  a  bleeder. 


Figure  5.  Bone  Chips  and  Bleeder 

The  ethmoid  sinuses  consist  of  hollow  cells  made  of  eggshell-like  bone.  The  bulla  is  the 
prominent  anterior  cell.  Simulated  dissection  occurs  within  the  dissection  volume  of  the 
surgical  instrument,  deleting  vertices  and  their  associated  polygons  and  exposing  the 
internal  surface  of  the  cell.  Figure  6  depicts  the  microdebrider  dissecting  the  ethmoid  bulla. 
The  far-right  image  shows  additional  ethmoid  cells  exposed  by  removal  of  the  entire  bulla. 


Figure  6.  Dissection  of  Ethmoid  Bulla 

The  approach  for  solid  structures,  such  as  the  uncinate  process,  displaces  rather  than 
removes  dissected  vertices.  When  dissection  occurs,  patient  model  vertices  move  from  inside 
the  instrument's  dissection  volume  to  outside.  The  displaced  vertices  receive  new  positions, 
normals  and  colors  appropriate  to  the  dissected  anatomy.  A  simple  non-anatomical 
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geometric  object  constrains  vertex  displacement  and  defines  vertex  motion  (Figure  7).  Figure 
8  shows  dissection  of  the  uncinate  with  a  left-side  back-biting  forceps  (surrounding  anatomy 
has  been  removed  for  clarity).  Vertices  in  the  uncinate  move  in  a  direction  that  extends 
radially  from  the  center  of  a  sphere.  The  sphere's  radius  limits  vertex  displacement,  ensuring 
that  dissection  causes  no  discontinuity  between  the  uncinate  and  adjacent  anatomical 
structures. 


Figure  7.  Tissue  Mass  Dissection 


Figure  8.  Uncinate  Dissection 

Incision  with  a  knife  or  scissors  is  similar  to  dissection.  For  most  anatomical  structures,  an 
incising  volume  is  a  simplified  model  that  approximates  the  instrument's  geometry. 
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However,  in  the  case  of  the  uncinate  or  turbinate,  a  plane  separates  vertices  that  connect  to 
adjacent  anatomical  structures  from  those  that  do  not  connect.  Incision  of  these  structures 
removes  vertices  in  a  volume  that  starts  as  the  tool  dissection  volume  and  extends  away  from 
the  separation  plane  far  enough  to  encompass  the  extent  of  the  structure's  tissue. 

Because  the  sinus  anatomy  is  rigid  to  good  approximation,  there  is  only  a  limited  need  for 
modeling  tissue  deformation.  Medialization  of  the  middle  turbinate  to  gain  access  deeper 
into  the  sinuses  is  the  only  maneuver  requiring  deformation.  The  simulator  employs  a 
simple  algorithm  to  displace  vertices  representing  this  fragile  bony  structure.  A  separation 
plane  divides  the  turbinate's  vertices  into  those  that  move  and  those  that  remain  stationary, 
preserving  the  integrity  of  the  overall  patient  model.  Movement  of  each  vertex  is  a  function 
of  its  distance  from  the  separation  plane.  Movable  vertices  farther  from  the  plane  undergo 
more  displacement  than  those  adjacent  to  the  plane.  The  algorithm  constrains  turbinate 
movement  to  where  the  turbinate  does  not  contact  adjacent  structures.  A  realistic  finite- 
element  approach  would  have  been  costly  in  computational  resources  and  is  not  necessary 
for  training.  The  simu-lator  does  not  model  surface  depression  via  instrument  palpation 
because  of  computational  resource  constraints  and  lack  of  training  value. 

Injection  of  a  vaso-constrictor  reduces  bleeding,  a  required  step  in  sinus  surgery.  The 
simulator  employs  recursive  algorithms  to  find  vertices  inside  the  needle's  injection  volume, 
averaging  their  original  color  with  blood  red  to  simulate  the  puncture.  Vertices,  in  a  larger 
volume  surrounding  the  needle's  tip,  mix  with  white  to  simulate  the  blanching.  Figure  9 
depicts  an  injection  in  progress. 


Figure  9.  Injection  of  the  Nasal  Passage 
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The  Simulator  -  Patient  Model  Development  -  Overview 

The  intricate  anatomy  of  the  human  head  and  neck  is  a  demanding  region  to  master.  A 
computer-generated  surface  that  accurately  represents  the  sinuses  requires  hundreds  of 
thousands  of  polygons,  stressing  current  technology.  Texture,  shading  and  sophisticated 
lighting  that  moves  in  real  time  with  the  eyepoint  add  to  the  complexity  of  rendering 
hardware.  Light  source  motion  precludes  preprocessing  of  highlights  so  that  the  rendering 
pipeline  must  calculate  them  in  real  time. 

The  patient  model  development  process  that  created  a  3D  sinus  model  for  use  in  die 
simulator  was  influenced  by  two  opposing  goals: 

a.  Fidelity:  The  model  must  accurately  represent  the  sinuses,  providing  a  "realistic" 
training  experience. 

b.  Efficiency:  The  simulation  computer  must  be  able  to  render  the  model  in  under  30 
ms  so  that  it  responds  quickly  to  student  actions. 

The  National  Library  of  Medicine  (NLM)  Visible  Male  data  set  formed  the  basis  for  deriving 
the  simulator's  patient  models.  A  total  of  1.7  GB  of  data,  consisting  of  magnetic  resonance 
(MR),  CT,  and  color  cryoslice  photographs,  comprise  the  record  of  the  subject's  head  and 
neck. 

NLM  axial  CT  images  were  the  source  for  the  coronal  CT  images  displayed  in  the  simulator 
and  also  for  the  cube  of  CT  densities  used  to  determine  the  stiffness  of  contacted  tissue. 
Processing  of  these  forms  of  the  patient  model  is  trivial  and  discussed  no  further  here. 

The  development  tools  and  processes  that  create  the  renderable  patient  model  underwent 
iterative  refinement,  improving  both  patient  model  fidelity  and  performance.  These 
paragraphs  focus  on  the  three  methods  that  comprise  the  renderable  model  development 
process: 

•  Segmentation 

•  Surface  Extraction 

•  Surface  Simplification 

The  Simulator  -  Patient  Model  Development  -  Segmentation 

The  set  of  NLM  2048  x  1216  x  24  bit  cryoslice  images  was  the  source  for  reconstructing  the 
patient  topology.  These  images  contain  higher  spatial  resolution,  0.33  x  0.33  x  1.0  mm,  than 
the  CT  or  MR  images. 

The  first  step  in  die  process  used  SGTsfrombin  utility  to  convert  the  cryoslice  images  from 
NLM  format  to  SGI  red,  green,  blue  (RGB)  format.  Using  SGI's  subimg  routine,  investigators 
then  cropped  the  original  images  to  focus  on  the  sinuses,  reducing  each  image  to  a 
manageable  351  x  426  x  24  bits.  Figure  10  depicts  a  cropped  cryoslice  image. 


19 


DAMD17-95-2-5023 
Final  Report 

The  subject's  nose  is  at  the  bottom  of  the  screen;  the  blue  areas  are  hollow  sinus  cavities. 
After  cropping  the  cryoslice  images  and  discarding  images  above  the  frontal  sinuses  and 
below  the  palate,  investigators  still  had  57  MB  of  source  data  to  manage. 


Figure  10.  Cropped  Cryoslice 


Investigators  encountered  significant  problems  extracting  surfaces  from  the  NLM  data.  An 
apparent  side  effect  of  freezing,  swollen  nasal  mucosa,  obscures  the  normal  anatomy  of  the 
osteomeatal  complex.  Also,  the  resolution  was  insufficient  to  capture  the  finer  details  in  the 
sinuses. 

Pathology  in  the  right  nasal  passage  complicated  matters  further.  A  large  polyp  exists 
beneath  the  middle  turbinate.  Additional  polyps  appear  near  the  superior  margin  of  the 
nasal  cavity  and  inside  the  ethmoid  air  cells.  Therefore,  the  subject's  left  side,  and  mid-line 
structures,  such  as  the  sphenoid  sinus,  became  the  basis  for  the  patient  model. 

To  perform  image  segmentation,  investigators  used  a  segmentation  program  written  at  The 
Ohio  State  University  (OSU)  consisting  of  neural  networks  using  oscillators.  This  program. 
Graph  -  Locally  Excitatory  Globally  Inhibitory  Oscillator  Network  or  G-LEGION  (26-27).  G- 
LEGION  combines  the  functionality  of  neural  networks  with  region  growing.  It  groups 
pixels  locally,  using  the  difference  between  neighboring  values. 
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As  with  all  region-growing  algorithms,  G-LEGION  tends  to  merge  neighboring  regions  that 
have  similar  characteristics.  Where  merging  occurred  improperly,  investigators  bound  the 
problem  with  a  three-dimensional  box  and  resegmented  the  bound  volume  using  a  new 
threshold,  tailored  to  solve  the  specific  problem. 

The  area  of  the  skin  just  inferior  to  the  root  of  the  nose  proved  difficult  because  of  noise  in 
the  surrounding  fixing  medium.  G-LEGION  had  trouble  defining  a  sharp  boundary.  Inves¬ 
tigators  again  used  restrictive  thresholding  to  segment  a  smooth  skin  surface  (Figure  11). 


Figure  11.  G-LEGION  Results 


G-LEGION  results  were  promising.  The  algorithm  performed  well  using  data  that  contained 
noise  and  ill-defined  boundaries.  However,  G-LEGION  did  not  provide  a  finished  product. 
The  G-LEGION  software  left  noise  in  the  segmented  image  and  failed  to  completely  segment 
some  regions.  Also  G-LEGION  could  not  automatically  correct  problems  with  swollen 
tissues  and  lack  of  resolution  in  the  data  set.  These  required  manual  intervention. 

G-LEGION  creates  segmentation  masks  which  are  images  that  have  a  unique  color  for  each 
image  segment.  For  example,  in  Figure  12,  the  middle  turbinate  is  dark  blue  and  the  nasal 
airway  is  yellow.  To  enable  manual  refinement  of  the  segmentation  masks,  investigators 
created  Editmask  using  the  SGI  ImageVision  Library.  Investigators  load  original  cryoslice 
files  and  tint  them  with  Editmask,  utilizing  G-LEGION  segmentation  results.  They  are  able 
to  zoom  into  problem  areas  and  page  up  and  down  through  the  slices.  Editmask  enables 
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segment  mask  editing  through  mouse-driven  painting  and  erasing.  Another  feature,  useful 
for  removing  noise,  allows  investigators  to  create  and  fill  polygons. 


Figure  12.  Hand-Edited  Segmentation  Masks  and  Tinted  Cryoslice  Image 


Limitations  in  the  NLM  resolution  are  apparent  in  several  structures  including  the  uncinate 
and  the  maxillary  ostium.  Therefore,  investigators  had  to  "improve"  the  data  by  adjusting 
the  segmentation  masks  to  create  a  more  realistic  depiction  of  the  structures  in  the 
reconstructed  topology. 


Raw  data 
I 

Convert  to  SGI  RGB  images 
(frombin) 

I 

Crop  images 
(subimg) 

I 

Segment  images 
(G-LEGION) 

I 

Hand-edit  segmentation  masks 
(Editmask) 

I 


Segmentation  Process  Flow 
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The  Simulator  -  Patient  Model  Development  -  Surface  Extraction 

For  surface  extraction,  investigators  downloaded  the  Visual  Tool  Kit  (VTK)  from  General 
Electric's  web  site.  This  freeware  provides  a  Marching  Cubes  (28)  method  to  extract  polygons 
from  volumetric  sets. 

The  first  step  in  this  process  is  to  convert  all  segmentation  masks  to  black  and  white 
(Color2BinMsk),  where  black  is  air  and  white  is  tissue.  This  allows  generation  of  a  single 
continuous  surface  that  comprises  all  segments.  Another  investigator-developed  tool, 
polymeshSplit,  separates  polygons  into  their  appropriate  segments. 

Recall  that  the  data  resolution  is  0.33  x  0.33  x  1.0  mm.  The  resolution  is  not  identical  for  all 
axes,  causing  banding  to  appear  in  the  resultant  polygonal  model.  To  address  the  problem, 
investigators  added  slices  via  interpolation  so  that  the  Marching  Cubes  software  would  have 
volumetric  data  with  identical  resolution  in  all  axes.  Investigators  used  SGI's  Image  Vision 


Convert  appropriate  color  segments  to  black  and  white 
(Color2BinMsk) 

I 

Reverse  order  of  images 
(IrevOrder) 

I 

Interpolate  between  slices 
(interp3D) 

I 

Blur  binary  masks  in  3D 
(blur3D) 

I 

Rotate  3D  volume  by  90  degrees 
(rgb2vdf,  vdf2rgb) 

I 

Rotate  images  by  270  degrees 
(iflip) 

I 

Convert  to  Volume  Data  File 
(rgb2vdf) 

I 

Convert  to  format  used  by  VTK 
(vdf2LorSlice) 

I 

Generate  tagged  vdf  for  use  with  polymeshSplit 
(vdftag) 

I 

Extract  surfaces 

(vtkVolumel6Reader;  vtkSliceCubes;  vtkMCubesReader;  vtkCleanPolyData; 

vtkPolyNormals;  vtkPolyWriter) 


Surface  Extraction  Process  Flow 
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Library  to  develop  interpolation  software  (interp3D).  Interpolation  occurs  in  one  dimension 
using  corresponding  pixels  in  the  user-specified  number  of  neighboring  images. 
Investigators  developed  two  interpolation  functions.  One  applies  equal  weight  to  all 
samples.  The  other  uses  a  Lagrangian  function  that  approximates  a  polynomial  over  the 
sample  values.  The  Lagrangian  filter  delivers  noticeably  better  results. 

The  Marching  Cube  algorithm  computes  normals  using  local  variations  in  the  surface, 
causing  wild  fluctuations.  This  approximation  is  unacceptable  when  the  rendering  pipeline 
uses  normals  for  shading.  Using  the  ImageVision  Library,  investigators  developed  3D 
blurring  software  that  inputs  the  volume  of  masks  and  blurs  the  data  (blur3D).  Investigators 
implemented  a  3D  box  filter  and  Gaussian  filter.  To  handle  the  edge  of  the  volume,  they 
reflected  the  images,  extending  the  volume.  Although  not  optimal,  this  approach  provided 
useful  results.  The  software  uses  a  command  line  interface  for  easy  insertion  into  the 
processing  pipeline. 


The  Simulator  -  Patient  Model  Development  -  Surface  Simplification 

Investigators  wrote  the  polymeshSplit  algorithm  to  separate  polygons  into  functional  groups. 
The  algorithm  checks  each  vertex  for  inclusion  into  one  or  more  patient  segments.  Some 
belong  to  adjacent  segments.  During  simplification,  the  smoothing  and  decimation  iterations 
rely  on  this  information  to  maintain  the  integrity  of  the  topology.  No  simplification  occurs 
along  the  boundary  between  adjacent  patient  segments. 

For  surface  simplification,  investigators  use  the  Laplacian  smoothing  software  based  on 
Taubin’s  work  at  IBM-Watson  (29).  This  method  reduces  the  high  frequency  in  the  geometry 
of  the  polygonal  mesh.  The  technique  suffers  in  that  excessive  smoothing  removes  smaller 
surfaces  and  desired  details. 


In  addition,  the  VTK  decimation  software  evaluates  each  vertex  for  local  planarity.  Upon 
meeting  certain  thresholds,  it  removes  the  vertex  and  modifies  the  local  topology.  This 
decreases  the  total  number  of  polygons  while  preserving  intricate  surface  details. 


The  vertex  coloring  program  (IvColorVert)  colors  an  Inventor  file  using  a  volume  of  color 
that  overlays  the  space  occupied  by  Inventor  file  polygons.  Since  real-time  performance  was 
a  key  objective  of  the  process,  investigators  generated  polygons  at  several  levels  of  fidelity: 


Low  resolution 
Medium  resolution 
High  resolution 
Very  high  resolution 


=  11680  triangles 

=  29401  triangles 
=  54650  triangles 
=  153987  triangles 


High  and  very  high  resolution  data  sets  contain  more  surface  detail  and  richer  color,  but  do 
not  deliver  real-time  performance.  Therefore,  the  simulator  uses  a  combination  of  low  and 
medium  resolution  components.  Investigators  partially  overcame  shortcomings  in  the  lower- 
fidelity  data  by  applying  texture  to  the  surface  using  a  commercially  available  modeling  tool, 
MultiGen.  For  texturing,  the  graphics  hardware  processes  a  gray-scale  texture,  modulating 
shaded  colors  at  and  between  the  vertices.  This  adds  surface  pattern  detail  between  vertices. 
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MultiGen  allowed  investigators  to  contort  various  anatomical  structures,  creating 
pathological  variants.  For  example,  using  MultiGen,  investigators  modified  the  standard 
middle  turbinate  to  create  both  a  paradoxical  turbinate  and  a  concha  bullosa. 


Split  polygons  into  patient  components 
(polymeshSplit) 

I 

Smooth  and  decimate  polygons 

(vtkPolyReader;  vtkSmoothPolyFilter;  vtkDecimate;  vtkCleanPolyData;  vtkPolyNormals; 

vtkTransformPolyFilter;  vtkPolyWriter) 

I 

Convert  VTK  format  to  Inventor  format 
(VtkToIv) 

I 

Color  Inventor  vertices  with  color  volume 
dvColorVert) 

I 

Add  texture  and  create  variants 
(MultiGen) 


Surface  Simplification  Process  Flow 

The  Simulator  -  Patient  Models 

The  standard  patient  model  consists  of  approximately  25,000  polygons.  The  model  is  a 
realistic  representation  of  the  skin,  nasal  passage,  turbinates,  ethmoid  sinuses,  maxillary 
sinuses,  and  the  sphenoid  sinuses.  Segmentation  of  these  anatomical  structures  is  the  first 
level  of  organization  in  the  patient  database.  For  example,  the  uncinate  is  in  its  own  file. 
Each  structure  belongs  to  a  software  class  that  models  its  behavior  under  surgical  interaction. 
Surgically  relevant  anatomical  segments  include  the  ethmoid  cells,  uncinate  process, 
maxillary  ostium,  sphenoid  ostium  and  middle  turbinate.  Segmentation  also  permits  scoring 
and  allows  for  easy  substitution  among  the  patient  variants.  Figure  13  shows  the  skin. 

At  load  time,  simulation  software  further  organizes  each  patient  model  segment  spatially. 
This  allows  the  graphics  rendering  software  to  traverse  only  that  subset  of  the  database 
likely  to  be  within  the  instantaneous  view  of  the  endoscope.  The  culling  of  the  database 
greatly  improves  performance  by  reducing  the  number  of  polygons  handled  by  the 
rendering  pipeline. 

The  simulator  uses  pre-processed  patient  models  in  three  forms.  First,  the  instructor 
interface  retrieves  and  displays  coronal  CT  images  stored  in  a  generic,  eight-bits-per-pixel, 
row-ordered  format.  Second,  simulation  software  reads  a  lower  resolution  cube  of  CT 
images  for  determining  stiffness  of  encountered  tissue.  Third,  the  rendering  and  surgical 
interaction  software  use  the  result  of  the  patient  model  development  process,  which  is  a 
geometric  representation  of  the  anatomy  in  Open  Inventor  format.  Vertices,  surface  normals, 
and  texture  coordinates  comprise  the  geometric  data  used  in  the  rendering  process. 
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Figure  13.  Skin  Model 

Along  with  the  standard  "healthy"  patient,  there  are  variants  that  enrich  the  student's 
training  experience.  To  enhance  military  readiness,  surgeons  use  the  trauma  model  shown 
in  Figure  14.  The  model  contains  a  low-velocity  projectile,  such  as  a  piece  of  shrapnel,  that 
enters  through  the  left  cheek  near  the  eye  and  lodges  in  the  maxillary  sinus  cavity.  A  case 
performed  by  Dr.  Gregory  Wiet  from  OSU  served  as  the  real-world  example  on  which  to 
base  this  model.  Robert  Eisler's  full  report  on  trauma  to  the  sinus  area  is  included  here  as 
Appendix  D.  The  figure  shows  the  entrance  wound  and  the  bullet  being  removed  by  an  up- 
biting  forceps.  Damage  to  various  anatomical  structures  is  visible  in  the  endoscope.  The 
surgeon  can  use  the  endoscope  to  traverse  the  shrapnel's  path  through  the  skin. 


Figure  14.  Trauma  Model  -  Shrapnel  Extraction 

Variants  to  the  standard  and  trauma  models  can  exist  alone  or  in  combination.  Among  these 
are  a  deviated  septum,  concha  bullosa,  paradoxical  turbinate  and  polyps.  Surgical  treatment 
on  the  simulator  is  possible  for  all  but  the  deviated  septum.  The  simulator  contains  reflected 
versions  of  all  models  to  enable  both  left  and  right  nostril  training. 
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The  Simulator  -  Virtual  Instruments 

The  simulator  provides  a  palate  of  15  surgical  instruments  (Figure  15).  Investigators  used 
SGI's  Interactive  Three-Dimensional  Modeler  (I3DM)  to  develop  the  instrument  models,  the 
source  data  for  which  was  actual  instruments  and  images  on  videotape. 

For  each  instrument,  a  Performer-loadable  file  specifies  the  visible  geometry,  emissive  color, 
reflective  color,specular  highlight  color,  and  shininess.  A  second  file  defines  the  simplified, 
non-rendered  geometry  for  haptic  and  surgical  interactions.  Each  instrument's  software  is  an 
object  derived  from  the  base  surgical  instrument  class,  facilitating  unique  instrument 
behaviors. 

To  understand  the  need  for  these  instruments,  consider  the  following,  typical  procedure. 
After  exploring  the  anatomy,  the  surgeon  uses  a  straight  needle  to  inject  surfaces  of  the 
anatomy  that  are  nearly  perpendicular  to  the  inserted  instrument.  Then,  the  surgeon  bends 
the  needle  for  injection  of  surfaces  that  are  nearly  parallel  to  the  shaft  of  the  instrument.  The 
injected  vaso-constrictor  reduces  bleeding.  The  surgeon  then  medializes  the  middle 
turbinate  with  the  freer  to  gain  access  deeper  into  the  sinuses.  If  the  turbinate  is 
pathological,  the  surgeon  may  dissect  it  with  scissors.  The  uncinate  process  is  dissected  with 
a  left  or  right  side  back-biting  forceps  (depending  on  which  nostril),  or  a  sickle  knife.  Blood, 
tissue,  and  bone  fragments  are  removed  with  the  suction  instrument.  Using  straight  or  up- 
biting  forceps,  the  surgeon  removes  other  tissue  and  bone  fragments  and  then  punches 
through  the  ethmoid  bulla,  removing  more  tissue.  The  microdebrider  is  used  to  clean  out 
the  remainder  of  the  ethmoid  cells.  Proceeding  to  the  maxillary  ostium,  the  surgeon  uses  a 
circular,  left  or  right  antrum  punch  to  perform  a  maxillary  antrostomy. 
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Figure  15.  Instrument  Palette 
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The  Simulator  -  Computer- Assisted  Instruction 

One  appealing  aspect  of  computer  simulation  is  the  potential  to  educate  students  and 
automatically  assess  their  performance.  Novice,  intermediate,  and  advanced  training  levels 
require  unique  training  aids.  The  simulator  has  built-in  aids  for  each  level  that  guide  and 
score  student  actions.  At  the  advanced  level,  the  student  performs  a  limited  anterior 
ethmoidectomy  without  prompting. 

Among  the  simulator  training  aids  are  aligned  successions  of  brightly  colored  hoops, 
forming  paths  through  which  the  student  must  maneuver  the  endoscope  for  a  training  score. 
The  paths  guide  the  examination  of  the  patient  prior  to  surgical  interaction  with  the  patient 
anatomy.  While  paths  for  all  training  levels  are  identical,  the  novice  level  lacks  the  context  of 
patient  anatomy.  Upon  completion  of  the  examination,  targets  in  the  form  of  bull's-eyes 
appear,  prompting  the  student  to  begin  the  injection  phase.  At  the  end  of  the  injection  phase, 
a  sequence  of  spherical  masses  provides  novice-level  dissection  training,  one  mass  for  each 
of  the  dissecting  instruments.  When  a  marker  is  completely  removed,  the  student  receives  a 
score  of  100%  for  that  marker.  Scores  are  reduced  if  the  action  exceeds  the  recommended 
time. 

Figure  16  shows  the  novice  training  environment  through  the  endoscope.  Red  capsules  with 
white  caps  provide  rich  motion  and  attitude  cues. 


Figure  16.  Novice  Model 
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As  shown  in  Figure  17,  intermediate  students  interact  with  identical  hoops  and  targets  that 
now  appear  in  the  context  of  sinus  anatomy.  The  dissection  task  at  this  level  transitions  from 
novice-level  spherical  masses  to  more  realistic  representations  of  patient  anatomy.  Labels 
appear  near  anatomical  structures  to  help  students  identify  dissectable  components  and 
provide  landmarks.  Dissection  scores  reflect  the  percentage  of  vertices  removed  from 
dissectable  components. 


Figure  17.  Intermediate  Model 


Figure  18  depicts  the  advanced  model.  Advanced  students  see  no  hoops,  although  invisible 
hoops  score  their  actions.  No  targets  guide  injection;  rather,  scores  reflect  student  injection 
of  appropriate  sinus  structures.  The  labels  are  invisible,  but  instructors  may  activate  them  if 
the  student  requires  assistance.  Advanced-level  dissection  is  identical  to  intermediate  level 
dissection. 

Audible  cues  announce  progress  toward  completion  of  the  procedure  and  suggest  the  next 
course  of  action.  As  a  student  successfully  negotiates  each  hoop,  the  simulator  announces 
"Hoop,"  providing  instantaneous  positive  feedback.  Upon  completing  the  examination,  the 
student  is  advised  by  the  simulator  to  "Begin  injection."  The  simulator  tracks  progress 
through  the  injection  with  audible  cues,  such  as,  "Middle  turbinate  injected  25%."  As  he 
successfully  completes  the  injection,  the  student  hears  "Begin  dissection."  Announcements 
of  progress  continue  during  dissection  of  the  various  anatomical  features,  e.g.,  "Uncinate 
dissected  25%. " 
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Figure  18.  Advanced  Model 


The  simulator  records  student  inputs  frame-by-frame,  precisely  auditing  student  actions. 
This  stored  data  is  useful  for  student  evaluations,  as  described  later  in  this  report.  The 
simulator  features  a  playback  capability  in  which  it  ignores  haptic  system  inputs  and  plays 
back  the  previously  recorded  student  and  instructor  actions  to  recreate  the  procedure. 
Playing  back  expert  actions  conveniently  demonstrates  surgical  procedures  for  student 
training  prior  to  simulator  use. 

At  the  time  the  proposal  was  written,  the  development  team  intended  to  develop  a  stand¬ 
alone,  interactive  tutorial  in  a  question-and-answer  format.  Dr.  Edmond  compiled  150 
questions  to  include  in  the  course;  however,  he  chose  to  abandon  the  development  of  the 
tutorial  in  favor  of  the  simulator-based,  computer-aided  instruction  described  above. 
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EVALUATION 

The  following  sections  contain  significant  portions  of  the  formal  simulator  evaluation; 
Appendix  E  contains  the  complete  evaluation.  The  evaluation  was  performed  by  the  Human 
Interface  Laboratory  completely  independently  of  the  development  effort.  For  this  project, 
evaluations  were  of  two  general  types: 

•  "Formative"  evaluation,  which  provided  design  specification  input  to  the 
development  team 

•  "Summative"  evaluation,  which  assessed  the  success  of  that  effort  by  formally 
analyzing  the  system's  effectiveness 

Evaluation  -  Formative 

Throughout  the  simulator's  development,  the  evaluation  team  worked  in  close  collaboration 
with  Dr.  Charles  Edmond  to  perform  the  formative  evaluation  and  make  design  recommen¬ 
dations  to  the  development  team. 

A  considerable  amount  of  effort  was  involved  in  establishing  a  "curriculum  wrapper"  during 
the  development  phase.  Investigators  initially  envisioned  the  simulator  embedded  within  a 
multimedia  training  system  that  would  provide  a  meaningful  context  and  set  the  protocol  for 
the  trainee. 

The  development  team  elected  instead  to  integrate  relevant  aspects  of  the  emerging  ENT 
curriculum  into  the  simulator  itself  and  to  postpone  further  efforts  to  develop  a  total 
curriculum  "package."  The  reasons  for  this  included: 

•  Development  of  a  full-blown  ESS  curriculum  was  beyond  the  scope  of  this  phase  of 
the  project. 

•  It  became  apparent  that  a  staged  protocol  approach  was  necessary  to  make  the 
simulator  effective  in  a  training  context. 

•  Useful  techniques  emerged,  such  as  the  use  of  navigation  hoops  and  injection 
targets,  that  could  be  integrated  relatively  easily. 

Evaluation  -  Summative 

The  primary  goals  of  this  phase  of  the  evaluation  effort  were  to  validate  the  simulator's 
utility  as  an  ESS  training  environment,  assess  its  usability,  and  provide  feedback  to  the 
development  team. 

Evaluation  -  Subjects 

Investigators  solicited  subjects  horn  three  groups:  non-physicians,  non-ENT  physicians,  and 
ENT  staff  and  residents.  Investigators  these  subjects  to  establish  a  baseline  and  asymptote 
for  the  evaluation  of  the  efficacy  of  the  simulator.  The  non-ENT  groups  allowed 
investigators  to  "shake  down"  the  system  and  research  protocols  without  utilizing  valuable 
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ENT  resident  subjects.  They  also  provided  baseline  scores  for  untrained /unfamiliar 
subjects.  The  non-ENT  physician  group  also  provided  investigators  with  valuable  input 
about  the  extensibility  of  the  simulator  to  non-ENT  applications. 

Twelve  volunteers  from  the  University  of  Washington  College  of  Engineering  comprised  the 
non-physician  group.  Subjects  ranged  in  age  from  23  to  54  and  included  graduate  students, 
professional  staff  and  faculty.  All  had  some  experience  with  simulation  and  virtual  reality. 
None  had  previously  used  an  endoscope  or  attended  medical  school. 

The  non-ENT  physician  group  consisted  of  eight  University  of  Washington  physicians  from 
specialties  other  than  otolaryngology.  This  group  provided  extensive  feedback  on  the 
simulator  design  and  utility  for  other  medical  and  surgical  tasks,  and  included  three 
videoendoscopic  surgeons,  two  radiologists,  one  neurosurgeon,  one  cardiologist,  and  one 
anesthesiologist.  Four  of  these  subjects  also  provided  investigators  with  complete  trial 
performance  data. 

The  ENT  group  comprised  twelve  staff  and  resident  ENTs  from  MAMC:  four  staff,  three 
second-year,  one  third-year,  two  fourth-year  and  two  fifth-year  residents.  Note  that  in  this 
report,  investigators  combined  the  single  third-year  resident  with  the  two  fourth-year 
residents  to  ensure  confidentiality. 

Evaluation  -  Performance  Criteria 

The  scoring  algorithm  for  Version  1.2  of  the  simulator  takes  into  consideration  both  accuracy 
and  time.  The  endoscopic  surgical  procedure  comprises  navigation,  injection  and  dissection 
tasks.  The  overall  trial  score  is: 

overall  score  =  (navigation  score  +  injection  score  +  dissection  score) ft 
where  task  scores  are: 

task  score  =  accuracy*optimal  time/completed  time 

Evaluation  Results  -  Non-Physicians 

Although  not  the  target  domain  audience  for  this  simulator,  the  non-physician  group 
provided  an  opportunity  to  look  at  the  difficulty  of  the  simulator  for  domain-naive  users  and 
to  further  work  out  requirements  for  instructional  presentation,  proctoring,  and  pacing  of  the 
"curriculum."  Since  the  novice  model  did  not  require  detailed  knowledge  of  the  surgical 
procedure  or  interaction  with  the  complex  paranasal  sinus  anatomy,  it  was  felt  that  this 
group  should  be  able  to  complete  the  trials. 

The  inherent  difficulty  of  the  task  for  inexperienced  users  is  evidenced  by  attrition  and 
initial  trial  scores.  Indeed,  complete  trial  data  are  available  for  only  nine  of  the  12  subjects  in 
this  group.  Of  the  three  lost  to  attrition,  one  terminated  the  session  due  to  disorientation  and 
discomfort,  and  two  were  unable  to  complete  even  the  navigation  task  during  the  time 
period  available  for  testing.  They  were  not  able  to  acquire  the  basic  skill  necessary  to  control 
the  positioning  of  the  virtual  endoscope. 
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Of  the  nine  remaining  subjects  in  this  group,  five  had  considerable  experience  with 
videogames  and  three  with  commercial  flight  simulators.  Thus,  the  trial  scores  for  this 
group  probably  represents  the  high  end  of  non-physician  users  and  should  not  be  taken  as  a 
sample  from  the  general  non-physidan  population. 

Despite  attrition  of  the  low-end  performers  and  the  unusual  prior  experience  of  the 
remaining  subjects,  the  non-physician  group  performed  significantly  worse  than  the  ENT 
group  on  their  initial  trials.  Unpaired  students'  t  comparisons  with  the  ENT  group  revealed 
significantly  lower  overall  trial  scores  for  novice  (non-physidan  mean  =  45.9,  ENT  mean  = 
65.1,  t  =  2.87,  p  =  .0098)  and  intermediate-level  training  (non-physidan  mean  =  53.7,  ENT 
mean  =  76.7,  t  =  3.53,  p  =  .0041). 

Six  of  the  12  subjects  in  the  non-physidan  group  had  complete  trial  scores  for  at  least  two 
novice  trials.  Students'  t  comparisons  of  their  first  and  second  novice  trials  showed  that  their 
second  trial  resulted  in  significantly  higher  trial  scores  (trial  1  mean  =  46.7,  trial  2  mean  = 
69.5,  t  =  2.33,  p  =  . 0423). 

Evaluation  Results  -  Non-ENT  Physicians 

The  primary  objectives  of  the  non-ENT  physidan  evaluations  were  to  provide  a  medically 
trained  comparison  and  "shakedown"  group  and  to  explore  the  perspectives  of  other  medical 
spedalties  regarding  potential  applications  for  this  sort  of  simulation  training.  In  particular, 
investigators  were  interested  in  applications  which  might  make  use  of  the  integrated 
approach  and  spedfic  components  incorporated  into  the  Madigan  ESS  simulator. 

Novice  performance  measures  for  the  non-ENT  physidan  group  were  very  similar  to  the 
those  for  the  experienced  ENT  group.  While  there  were  no  statistically  significant 
differences  between  these  two  groups,  the  mean  differences  for  all  of  the  performance 
measures  were  in  the  expected  direction.  This  finding  may  be  due  to  the  considerably  broad 
prior  medical  and  interface  experience  for  subjects  in  this  group. 

Three  of  the  non-ENT  physidan  subjects  also  tried  the  intermediate  model.  Comparison  of 
the  non-ENT  physidans  with  the  ENTs  on  their  first  trial  on  the  intermediate  model  reveals 
an  interesting  finding.  While  there  was  no  significant  difference  between  the  groups  in 
overall  score  and  trial  time,  the  non-ENTs  did  considerably  better  than  the  ENTs  on  the 
dissection  subtask  (non-ENT  mean  =  84.0,  ENT  mean  =  63.1,  t  =  2.439,  p  =  .0312). 
Investigators  expected  that  the  ENT  group  would  be  superior  to  all  other  groups  on  this  task. 

Perhaps  the  greatest  mitigating  factor  is  the  extensive  experience  that  two  of  these  four 
subjects  had  acquired  as  videoendoscopic  sturgeons.  Their  spedality  gives  them 
considerably  more  endocscopic  experience  than  ENT  physidans  generally  aquire.  General 
videoendo-scopic  procedures  are  more  challenging  from  a  psychomotor  perspective.  In 
addition  to  navigation,  injection  and  dissection,  their  procedures  may  require  retracting, 
cautery,  cutting,  suturing  and  true  dissection,  with  the  reflection  of  tissues  away  from  the 
area  of  interest.  Further,  they  perform  these  tasks  in  a  more  bi-dexterous  fashion;  the  ENT 
surgeon  uses  both  hands,  but  usually  one  at  a  time.  Finally,  the  videoendoscopic  surgeon's 


34 


DAMD17-95-2-5023 
Final  Report 

psychomotor  abilities  may  be  challenged  more  during  their  procedures,  because  the 
endoscope  and  instruments  are  usually  not  coaxial. 

Evaluation  Results  -  ENTs 

Simulator  performance  measures  were  acquired  on  53  separate  trials  for  the  12  ENT  subjects 
between  May  and  August  of  1997.  The  number  of  trials  acquired  for  each  subject  ranged 
from  2  to  12,  with  all  subjects  minimally  experiencing  both  the  novice  and  the  intermediate 
model. 

To  examine  the  degree  to  which  prior  OR  experience  might  contribute  to  these  initial  score 
differences,  investigators  calculated  the  Pearson  product-moment  correlation  coefficients 
("r")  across  the  primary  performance  measures  (trial  time  and  trial  score)  and  the  primary 
experience  measures  (age,  years  of  ESS  training,  and  approximate  number  of  actual  ESS 
procedures  performed).  A  matrix  of  these  correlations  appears  in  Table  1  (critical  r  =  .6021 
for  probability  p  <  .05,  degrees  of  freedom  df  =  9). 

As  can  be  seen,  the  best  predictor  of  novice  trial  time  from  among  the  surgical  experience 
measures  is  the  number  of  actual  ESS  cases  performed  (r  =  -.652),  with  trial  time  decreasing 
significantly  with  surgical  experience.  For  trial  score,  the  best  experiential  predictor  is  again 
the  number  of  ESS  cases  performed  (r  =  .526).  This  finding  suggests  that  the  simulator 
provides  a  valid  reflection  of  the  skills  acquired  in  ESS  procedures. 

Table  1.  Correlation  matrix  showing  relationships  among  ESS  experience  measures  and  overall 
performance  measures  on  novice  model  for  12  ENT  subjects. 


Score 

ESS  performed 
Years  training 
Age 

Trialtime 


1 

.526 

1 

.38 

.923 

1 

.236 

.662 

.713 

1 

-.966 

-.652 

-.514 

-.33 

1 

Note:  1  case  deleted  with  missing  values 

Figure  19  shows  the  breakdown  of  first-time  novice  (abstract)  model  scores  for  this  group. 
The  linear  relationship  between  overall  novice  trial  score  and  year  of  residency  is  apparent 
from  this  mapping.  It  is  interesting  to  note  that  the  R5  residents  appear  to  have  higher  scores 
on  the  novice  model  than  do  the  experienced  ENT  staff  subjects.  This  could  be  due  a 
number  of  factors,  including  the  relatively  high  number  of  procedures  being  performed 
routinely  by  the  R5s. 
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Figure  19.  Overall  Scores  for  First  Trial  on  Novice  Model 

Although  investigators  expected  a  positive  relationship  between  ESS  experience  and  novice 
model  performance,  they  anticipated  that  operating  room  experience  would  be  even  more 
predictive  of  performance  at  the  intermediate  level,  when  the  sinus  anatomy  serves  as  the 
model.  A  matrix  of  these  correlations  appears  in  Table  2  (critical  r  =  .6319  for  p  <  .05,  df  8). 
The  initial  performance  on  the  intermediate  model  is  positively  correlated  with  prior  ESS 
experience,  but  the  relationship  is  not  as  strong  as  expected.  This  may  relate  to  the  training 
effect  provided  by  the  simulator  in  the  novice  model. 

Table  2.  Correlation  matrix  showing  relationships  among  ESS  experience  measures  and  overall 
performance  measures  on  intermediate  model  for  12  ENT  subjects. 


Score 

FSS  perf  . 

Years  Tr 

Aae 

Trialtime  _ 

Score 

1 

ESS  Performed 

.555 

1 

Years  Training 

.356 

.931 

1 

Age 

.581 

.667 

.68 

1 

Trialtime 

-.795 

-.144 

-.014 

-.475 

1 

Note:  1  case  deleted  with  missing  values. 
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As  shown  in  Figure  20,  scores  on  the  initial  intermediate  trial  generally  improved  with  year 
of  residency. 
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Figure  20.  Overall  Scores  for  First  Trial  on  Intermediate  Model  for  ENT  Staff 

While  intermediate  trial  times  for  the  ENT  staff  might  be  expected  to  be  lower  than  those  for 
the  residents,  their  real-world  experience  with  the  procedure  may  actually  make  them  more 
cautious,  as  noted  above.  The  experienced  sinus  surgeon  knows,  for  example,  that  if  one 
traumatizes  the  septum  or  middle  turbinate  during  the  navigation  phase,  it  will  cause 
bleeding  that  will  affect  the  rest  of  the  surgical  procedure. 
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Figure  21.  Overall  Times  for  First  Trial  on  Intermediate  Model  for  ENT  Staff 
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Figure  22  shows  the  overall  scores  for  the  first  advanced  trial  by  subjects  in  the  ENT  group. 
Note  that  two  of  the  staff  subjects  did  not  attempt  the  advanced  model. 


Figure  22.  Overall  Trial  Scores  for  First  Trial  on  Advanced  Model  for  ENT  Staff 

Surprisingly,  initial  performance  on  the  advanced  model  revealed  a  tendency  for  the  more 
senior  residents  to  perform  at  about  the  same  speed  as  the  staff  ENT  subjects,  but  with 
slightly  lower  overall  scores.  This  may  reflect  the  ability  of  the  more  experienced  surgeons 
to  perform  maneuvers  with  a  higher  degree  of  accuracy.  Also,  since  there  is  no  strong 
correlation  between  initial  performance  at  the  advanced  level  and  experience,  one  may 
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Figure  23.  Injection  Scores  for  First  Trial  on  Advanced  Model  for  ENT  Staff 
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condude  that  the  novice  and  intermediate  level  experience  provided  its  own  training  effect. 
In  fact,  the  two  highest  resident  scores  were  registered  by  subjects  who  had  received 
additional  novice  and  intermediate-level  training. 

While  overall  time  and  score  for  staff  ENTs  did  not  appear  to  differ  uniformly  from  the 
resident  times  and  scores  on  initial  advanced  model  trials,  their  performance  on  the  injection 
subtask  appeared  to  be  superior  to  the  resident  group  (Figure  23).  Similarly  contrasting 
performance  was  not  seen  for  the  navigation  and  dissection  subtasks. 

Although  navigation  and  dissection  may  vary  with  individual  case  and  priorities,  the  data 
suggest  that  injection  efficiency  may  be  a  hallmark  of  the  experienced  sinus  surgeon.  This 
finding  should  be  followed  up  and  verified  in  future  evaluation  studies. 

Evaluation  Results  -  Asymptotic  Performance 

While  extensive  learning  curve  data  are  not  available,  repeated  measures  on  the  steady-state 
system  are  available  for  our  two  primary  evaluation  proctors,  an  ESS  surgeon  and  an 
engineering  student.  These  two  subjects  each  tested  the  simulator  in  various  degrees  of 
development  30-40  times  over  a  6  to  8  month  period.  Their  later  scores  may  provide  an 
estimate  of  asymptotic  performance  on  the  simulator. 

Figure  24  shows  simulator  performance  scores  for  the  last  11  trials  by  the  experienced  ENT 
proctor  and  the  engineering  student.  These  findings  suggest  that  extensive  experience  with 
the  simulator  may  afford  a  non-ENT  subject  performance  values  which  are  comparable  with 
an  experienced  ENT  proctor.  Assuming  validation  of  findings  discussed  above,  this  bodes 
well  for  the  training  effectiveness  of  the  simulator. 


Figure  24.  Comparison  of  Asymptotic  Trial  Scores  on  Each  Model  for  Non-ENT  Proctor 

and  Experienced  ENT  Proctor 
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Evaluation  -  Optimal  Path  Analysis 

Another  approach  for  evaluating  user  performance  is  to  look  at  the  paths  of  the  virtual  tools 
over  the  course  of  a  procedure.  Figures  25,  26,  and  27  show  the  performance  paths  of  three 
individuals:  a  non-physician,  a  second-year  resident,  and  a  fifth-year  resident,  respectively. 
Each  color  in  the  figure  represents  an  individual  surgical  tool  used  to  dissect  a  small  abstract 
sphere.  In  these  tracings,  the  control  shown  in  the  color  tracings  is  indicative  of  the  precision 
and  accuracy  of  the  subject  in  manipulating  the  tool's  position.  (The  dark  blue  lines  should 
be  disregarded,  because  they  trace  multiple  scattered  targets.) 

The  tracings  of  the  non-physician  are  depicted  in  Figure  25.  Figure  26  shows  the  tracings  of  a 
second-year  resident.  Notice  that  the  colors  show  smaller,  tighter  movements.  Cleary  this 
subject  performed  better  than  the  non-physician.  Figure  27  shows  the  performance  of  a  fifth- 
year  ENT  resident,  indicating  even  better  accuracy  and  precision. 

In  addition  to  drawing  the  paths  for  visual  analysis,  investigators  calculated  a  set  of  objective 
scores  based  on  the  tool  path  data.  The  measure  is  calculated  for  each  sphere  as 

sphere  path  score  =  total  frames  *  mean  tool  deviation  from  centroid  / 1000 

Note  that  the  lower  the  score,  the  better  the  performance.  First-order  correlation  between  the 
path  measure  and  the  original  simulator  score  correlates  with  a  value  of  -.798  (critical  r  = 
.4060  for  p  <.05,  df  22).  This  further  validates  the  scoring  algorithm. 


Figure  25.  Simulated  Dissection  Task  Performed  by  Non-Physician 
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RECOMMENDATIONS 

Several  areas  in  this  ESS  simulation  project  are  recommended  for  further  effort:  patient 
model  enhancements,  hardware  upgrades,  software  revisions,  haptic  refinements,  and 
training  effectiveness  evaluations. 

Patient  Model  Enhancements 

The  patient  model  should  be  refined.  Model  enhancements  should  revolve  around 
increasing  the  number  of  polygons  to  take  advantage  of  newer  hardware  and  segmenting 
additional  features.  New  anatomical  features  would  include  the  frontal  sinus,  frontal  recess, 
skull  base,  pituitary  gland,  and  normal  and  pathological  variations.  A  larger  library  of 
patient  models  would  allow  an  instructor  to  select  and  create  custom  models  to  increase 
student  familiarity  with  a  wide  range  of  conditions.  These  additions  would  also  allow 
procedures  for  sister  services  that  perform  endoscopic  techniques  in  the  nose,  such  as  an 
ophthalmological  dacrocystorhinostomy  or  a  neurosurgical  endoscopic  hypophysectomy. 

Hardware  Upgrades 

As  the  patient  model  becomes  more  complex,  the  computational  problem  grows. 
Investigators  made  significant  reductions  in  polygon  counts  to  meet  real-time  performance 
requirements.  The  computational  limitation  resulted  in  less  visual  realism  and  perceivable 
latency  when  operating  in  the  anatomically  dense  areas  of  surgical  dissection.  The 
acquisition  of  a  more  powerful  computer  would  render  more  complex  patient  models 
without  sacrificing  real-time  performance. 

Software  Revisions 

Model  enhancements  and  hardware  upgrades  require  software  modifications  to  support 
them.  Additionally,  clinicians  desire  an  interactive  training  tool  to  aid  student  instruction  by 
viewing  the  3-D  patient  model  in  a  perspective  that  is  not  available  in  the  real  world.  Further 
software  modifications  will  accommodate  anticipated  recommendations  as  the  simulator 
undergoes  field  tests. 

Haptic  Refinements 

Future  work  includes  enhancing  haptic  system  fidelity.  This  cooperative  agreement  did  not 
allow  sufficient  time  to  explore  the  more  challenging  aspects  of  force  feedback.  The  develop¬ 
ment  team  should  weigh  the  cost  of  technologically  immature  haptic  refinements  over  more 
perceptible  gains  in  the  other  areas  previously  mentioned.  At  the  end  of  this  contract, 
investigators  engineered  changes  to  the  haptic  system  to  improve  its  reliability.  There  is,  as 
yet,  no  data  to  judge  the  effectiveness. 
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Evaluations 

Simulators  have  not  yet  proven  themselves  to  increase  readiness  with  respect  to  basic 
surgical  skills.  Future  work  has  to  begin  to  explore  the  impact  of  experience  with  the 
simulator  on  actual  operating  room  performance.  Investigators  must  gather  the  data 
necessary  to  evaluate  training  effectiveness. 

Other 

Future  work  will  also  consist  of  monitoring  advances  made  by  associate  and  peer 
investigators  in  areas  of  soft  tissue  deformation,  volumetric  rendering,  finite  element 
modeling,  force  feedback  and  embedded  intelligence.  Surgical  simulation  is  emerging  from 
its  infancy,  and  its  fulfillment  will  occur  only  through  continued  support  from  the 
visionaries,  research  scientists,  and  clinicians. 
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CONCLUSIONS 

This  cooperative  agreement  enabled  the  development  of  a  state-of-the-art  training  device  for 
endoscopic  sinus  surgeons.  The  ESS  simulator  takes  medical  simulation  a  major  step 
forward  in  its  evolution.  Aside  from  its  technical  accomplishments,  the  integration  of  a  well 
thought-out  curricular  framework  allows  it  to  take  advantage  of  virtual  reality  without 
sacrificing  the  benefits  of  more  traditional  computer-aided  instruction.  Preliminary  simulator 
evaluation  results  support  the  notion  that  recent  gains  in  virtual  reality  technology  enable 
real  surgical  skills  and  procedure  awareness  training. 

The  validity  of  the  simulator  for  the  ESS  domain  is  suggested  by  a  number  of  findings: 

•  ENT  subjects  performed  significantly  better  than  non-physician  subjects  on  both 
the  novice  (abstract)  and  intermediate  (anatomical  with  aids)  models. 

•  Initial  performance  on  the  novice  model  correlates  with  residency  level  and  degree 
of  prior  ESS  experience. 

•  Patterns  of  difficulty  for  asymptotic  performance  on  the  simulator  seem  to  match 
the  typical  pattern  of  subtask  difficulty  in  the  operating  room. 

•  Subject  ratings  of  the  realism  of  the  virtual  anatomical  model  were  consistently 
high  on  the  post-session  questionnaire  and  in  open-ended  comments. 

•  Post-training  questionnaire  responses  confirm  that  the  ENT  subjects  perceived  the 
simulator  as  valid  and  useful  for  ESS  training. 

Our  responses  to  questions  put  forth  as  the  purpose  of  this  cooperative  agreement  follow: 

a.  Can  surgical  simulation  enhance  the  comprehension  and  performance  of  minimally  invasive 
surgical  procedures? 

Preliminary  data  indicates  that  the  simulator  can  enhance  comprehension  and 
performance  of  endoscopic  sinus  surgery.  However,  this  data  is  indirect. 
Continuation  of  studies  started  with  this  cooperative  agreement  will  yield  more 
direct  measurements. 

b.  Are  objective  performances  of  simulated  surgeries  useful  in  the  evaluation  of  surgical 
competence? 

While  the  evaluation  suggests  that  the  simulator  objectively  measures  surgical 
skills  such  as  speed,  precision,  dexterity,  and  spatial  perception,  there  are  other 
aspects  of  surgical  competence  such  as  cognitive  ability,  observations,  deductions, 
and  actions  that  require  further  study.  Continuation  of  studies  started  with  this 
cooperative  agreement  will  yield  more  definitive  results. 

c.  What  levels  of  visual  complexity  and  tactile  response  are  necessary  to  achieve  enhanced 
training? 

The  evaluation  suggests  that  the  simulator  is  valid  for  the  ESS  domain.  It  appears 
that  training  takes  place  using  the  level  of  visual  complexity  and  tactile  response  in 
version  1.2  of  the  simulation.  The  current  revision,  1.3,  incorporates  substantial 
improvements  in  tactile  response  that  should  provide  even  better  training. 
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d.  Can  the  above  objectives  be  achieved  at  an  acceptable  cost? 

Lockheed  Martin  configured  and  demonstrated  a  low-price  version  of  the 
simulator.  Initial  marketing  inquiries  show  the  price  of  that  configuration  to  be 
acceptable  to  many  in  the  medical  community. 
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Commercialization  Plan 

In  accordance  with  the  Statement  of  Work,  Lockheed  Martin  Tactical  Defense  Systems 
explored  the  feasibility  of  commercial  sales  of  the  Endoscopic  Sinus  Surgery  Simulator.  The 
initial  plan  to  commercialize  the  device  follows. 

Proposed  Product  Goals 

Lockheed  Martin  discussed  simulator  product  goals  with  clinical  consultants,  medical 
education  providers,  and  equipment  vendors.  The  initial  goals  of  this  product  are: 

•  The  simulator  should  provide  initial  and  continuation  training  for  endoscopic  sinus 
surgery  (ESS)  procedures. 

•  The  simulator  could  support  demonstrations  of  new  equipment  and  techniques 
offered  by  vendors  and  minimally  invasive  surgery  (MIS)  equipment 
manufacturers.  Manufacturers  could  also  use  the  device  to  support  product 
development. 

•  The  simulator  could  support  rapid  and  safe  experimentation  and  provide 
assistance  in  the  certification  of  new  procedures. 

•  Eventually,  with  the  addition  of  increased  functionality,  the  simulator  could 
support  "mission  rehearsal"  of  planned  procedures  utilizing  actual  patient  data. 

Problem  Statement 

Lockheed  Martin  investigated  current  otolaryngology  problems  suitable  to  the  application  of 
simulation  and  training  technology.  Analysis  focused  on  problems  encountered  by  practi¬ 
tioners,  health  care  providers,  equipment  manufacturers,  and  certifying  agencies.  Our 
analysis  yielded  the  following  issues  and  problems. 

Practitioners:  For  otolaryngology  practitioners,  there  are  two  primary  issues: 

•  Endoscopic  Sinus  Surgery  techniques  are  difficult  to  master,  involving  both 
cognitive  and  psycho-motor  skills. 

•  The  field  of  minimally  invasive  surgery  is  growing  rapidly,  introducing  new 
procedures  and  tools  at  a  fast  pace.  Current  teaching  methods  usually 
consist  of  a  weekend  seminar  by  the  equipment  manufacturer,  using  limited 
training  aids  to  learn  new  procedures. 

Health  Care  Providers:  There  are  several  areas  of  concern  to  hospitals  and  health  care  organ¬ 
izations: 

•  Currently,  there  is  no  method  to  verify  that  an  otolaryngologist  has  mastered 
the  psycho-motor  skills  necessary  to  perform  ESS  procedures.  Testing 
methods  consist  of  written  tests,  or  procedures  performed  on  an  animal. 

These  methods  evaluate  the  cognitive  aspects  of  the  procedure,  but  do  not 
evaluate  important  psycho-motor  skills. 
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•  There  is  no  efficient  means  to  quantitatively  measure  practitioner  skills  for 
clinical  privileging.  Also,  the  practitioner  who  does  not  perform  a  large 
number  of  these  procedures  may  not  keep  skills  current.  Lack  of  "practice" 
results  in  the  potential  for  serious  complications  to  patients,  additional  time 
in  the  operating  room,  and  an  increase  in  malpractice  suits. 

Surgical  Tool  Manufacturers:  Manufacturers  of  surgical  tools  and  associated  equipment 
require  training  and  simulation  technology.  The  Food  and  Drug  Administration  (FDA) 
requires  manufacturers  to  provide  practitioner  training  to  ensure  some  level  of  competence 
with  the  equipment  or  procedure.  Many  vendors  provide  training  with  weekend  seminars, 
often  using  crude  training  devices,  or  animals.  Surgical  equipment  manufacturers  have  no 
state-of-the-art  means  to  demonstrate  product  features  or  to  aid  in  the  marketing  of  the 
product.  A  high-fidelity  training  device  would  allow  simulation  of  the  new  product  features 
for  sales  assistance  and  potentially  for  gaining  FDA  approval. 

Certifying  Agencies:  No  means  currently  exist  to  quantitatively  measure  surgical  skill  in  a 
controlled  environment.  Organizations  such  as  the  American  College  of  Surgeons  could 
possibly  use  a  device  that  ascertains  and  registers  a  practitioner's  skill  level.  A  recent  report 
by  the  Technology  Marketing  Group,  Des  Plaines,  IL  indicates  that  otolaryngology 
minimally  invasive  surgery  procedures  in  the  US  will  grow  by  19%  annually. 

Proposed  Market 

The  market  for  an  ESS  simulation  device  may  be  divided  into  the  following  segments: 

•  Medical  education  and  teaching  programs 

•  Health  care  providers  and  Health  Maintenance  Organizations 

•  Large  otolaryngology  group  practices 

•  Equipment  manufacturers  and  vendors 

•  Oredentialing  boards,  continuing  medical  education  providers,  licensing 
boards,  malpractice  insurers 

Medical  Education  and  Teaching  Programs:  Current  ESS  training  consists  primarily  of 
classroom  instruction  and  practice  with  animals  and  cadavers.  The  proposed  training  device 
moves  training  from  the  laboratory  into  the  classroom,  allowing  "hands  on"  training  earlier 
in  the  curriculum.  There  are  126  medical  schools  and  270  teaching  hospitals  and  university 
medical  centers  in  the  United  States. 

Health  Care  Providers  and  Health  Maintenance  Organizations:  This  market  contains  strong 
potential  for  several  reasons: 

•  The  device  allows  practitioners  to  become  more  adept  by  practicing  procedures. 
This  would  result  in  better  patient  care,  less  time  in  the  operating  room,  and  higher 
patient  case  load. 

•  Use  of  the  simulator  would  ensure  that  a  practitioner  performs  a  minimal  number 
of  procedures  for  the  purpose  of  maintaining  surgical  skills. 
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•  The  combination  of  skills  maintenance  and  less  patient  time  in  the  operating  room 
favorably  impacts  the  cost  of  the  procedure  and  medical  malpractice  costs. 

Large  Otolaryngology  Group  Practices:  This  market  segment  benefits  for  the  same  reasons 
as  Health  Care  Providers  and  Health  Maintenance  Organizations.  Skills  maintenance  is  the 
primary  selling  point  for  large  group  practices. 

ESS  Equipment  Manufacturers:  The  FDA  is  moving  towards  having  manufacturers  and 
vendors  of  medical  equipment  become  responsible  for  training  practitioners  to  use  their 
equipment  and  techniques.  New  product  and  procedure  training  typically  consists  of  a 
weekend  seminar,  demonstrations  of  the  device,  videos,  and  instruction  involving  simplified 
training  devices,  animals  or  cadavers.  A  comprehensive  training  device  could  allow 
manufacturers  to  become  involved  in  credentialing. 

Credentialing  Boards,  Continuing  Medical  Education  Providers,  Licensing  Boards,  and 
Malpractice  Insurers:  This  market  segment  is  potentially  important  as  boards  become  more 
involved  in  ensuring  that  practitioners  qualify  to  perform  a  growing  number  of  MIS 
procedures  at  a  base  competency. 

Marketing  Approach 

This  plan  builds  upon  the  ESS  device  delivered  to  Madigan  Army  Hospital.  Lockheed 
Martin  will  continue  to  refine  the  simulator  based  on  comments  received  from  simulator 
evaluations.  Lockheed  Martin  has,  at  its  own  expense,  procured  a  second  device  for  use  in 
further  simulator  development  and  for  product  demonstration.  This  device  provides  a 
second  platform  for  continuous  product  improvement  as  well  as  the  capability  to 
demonstrate  the  capabilities  of  the  product  to  potential  customers.  It  is  anticipated  that  initial 
sales  will  come  horn  prestigious  hospitals  and  medical  schools.  Medical  equipment  vendors 
have  also  expressed  interest  in  the  product.  Lockheed  Martin  will  continue  to  provide 
product  demonstrations  that  incorporate  the  latest  product  improvements.  Lockheed  Martin 
is  discussing  long-term  marketing  services  for  this  product  with  a  firm  that  specializes  in  the 
marketing  of  medical  training  equipment. 
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1  INTRODUCTION 

This  report  represents  the  deliverable  for  Lockheed  Martin  Tactical  Defense  Systems  (formerly  Loral  Defense 
Systems-Akron),  Purchase  Order  6X0062  dated  15  March  1996.  The  effort  encompassed  100  person-hours  of 
effort  and  was  organized  into  three  tasks.  The  first  task  described  the  frequency  and  types  of  combat-related 
penetrating  facial  wounds,  focusing  on  paranasal  sinus  involvement.  In  the  second  task,  the  range  and  extent  of 
tissue  response  associated  with  these  wounds  is  documented.  In  the  final  task,  a  plan  leading  to  development  of 
computer-based  models  that  describe  tissue  disruption  from  combat-related  penetrating  facial  wounds  is 
presented. 

The  sources  of  data  employed  in  this  effort  were  threefold  and  consisted  of:  1)  an  extensive  review  of  data  from 
MEDLINE,  the  Defense  Technical  Information  Center,  and  wound  ballistics  literature  concerning  penetrating 
facial  injuries  from  ballistic  impact;  2)  personal  communication,  particularly  concerning  the  WDMET  database; 
and  3)  ongoing  wound  ballistics  research  at  the  Applied  Mechanics  and  Material  Sciences  Group  of  Mission 
Research  Corporation  (Fountain  Valley,  CA).  Major  conclusions  from  this  effort  are  summarized  below. 

Historically,  the  midfacial  region  is  the  most  vulnerable  region  in  the  body  for  combat-related,  penetrating 
wounds.  Combat-related,  penetrating  maxillofacial  wounds,  particularly  in  the  midface  (upper  lip  to 
supraorbital  ridge),  occur  far  more  frequently,  per  unit  projected  area,  than  any  other  region  of  the  body.  This  is 
a  trend  that  has  been  consistent  through  WWII,  the  Korean  War,  Vietnam  and  recent  regional  conflicts.  Missile 
wounds  of  the  maxillofacial  region,  which  represent  less  than  2%  of  total  projected  body  surface  area, 
accounted  for  approximately  15%  of  all  wounds  in  the  Vietnam  conflict.  Of  these  combat  wounds,  38%  were 
gunshot  wounds,  52%  were  fragment  wounds,  and  about  10%  were  other  types.1  Analysis  of  164  soldiers  killed 
in  the  Lebanon  War  with  Israel,  representing  405  penetrating  war  injuries  -  290  (72%)  shrapnel  and  115  (28%) 
bullet  -  showed  that  the  face  accounted  for  22.2  percent  of  all  penetrating  injuries.2  This  is  7.3  times  more  than 
its  proportion  of  total  body  area.  The  midface  from  the  level  of  the  lips  to  the  zygomatic  bones  was  the  most 
vulnerable  area  of  the  body,  taking  42  hits  (10%  of  all  penetrating  wounds). 

Combat-related  penetrating  facial  wounds  affecting  the  paranasal  sinuses  usually  occur  in  conjunction  with 
extensive  maxillofacial  or  craniofacial  wounds  caused  by  high-velocity  military  bullets  or  fragments  from  close 
proximity  to  exploding  mortar  and  artillery  shells.  These  wounds  typically  affect  large  areas  of  the  face,  often 
involving  avulsion  of  osseous  and  soft  tissue.  Civilian  handgun  and  combat-related,  low-velocity  fragmentation 
wounds  to  the  face  typically  involve  relatively  superficial  soft  tissue  penetration,  often  with  associated  fractures 
of  the  mandible  or  maxillary  complex.  In  contradistinction  to  high-velocity  missiles,  low-velocity  missiles  do 
not  commonly  cause  significant  vascular  trauma.  The  missile  will  commonly  push  nerves  out  of  its  pathway  as 
they  are  only  loosely  fixed  in  tissue  and  have  a  great  deal  of  elasticity.3  In  these  wounds,  the  missile  often 
remains  lodged  in  soft  tissue  and  can  involve  the  paranasal  sinuses.  Sinus  involvement  is  not  nearly  as  common 
in  these  wounds  as  with  high-velocity  wounds,  however.  This  is  probably  an  artifact  of  the  large  areas  affected 
by  the  high-velocity  military  bullet  or  high-velocity  fragmentation  injury  as  opposed  to  the  much  more  localized 
tissue  response  associated  with  the  low-velocity  handgun  or  low-velocity  fragmentation  injury. 


1  Rich,  N.M.,  "Evaluation  of  missile  wounds  at  the  2nd  Surgical  hospital  in  Vietnam,"  Plastic  and  Maxillofacial 
Trauma  Symposium,  Vol.  1.,  St.  Louis,  The  C.V.  Mosby  Co.,  1969,  p.  9. 

2  Gofrit,  O.N.,  Kovalski,  N.,  et  al.,  “Accurate  anatomical  location  of  war  injuries;  analysis  of  the  Lebanon  war  fatal 
casualties  and  the  proposition  of  new  principles  for  the  designing  of  military  personal  armour  system,”  Injury, 
27(8):  577-81,  1996. 

3  Holt,  R.G.,  “Wound  Ballistics  of  Gunshot  Injuries  to  the  Head  and  Neck,”  Arch.  Otolaryngol,  109:  313-18, 1983. 
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A  major  exception  to  this  generalization,  however,  is  among  U.S.  Army  aviators,  a  population  for  which  low- 
velocity  fragmentation  wounds  to  the  face  have  historically  been  a  common  form  of  penetrating  injury.  Among 
U.S.  Army  aviators  in  Vietnam,  the  face  accounted  for  over  half  of  head  and  neck  wounds.4  Secondary  debris 
from  armor  spall  and  fragmentation  of  transparent  enclosures  caused  most  of  these  wounds.  The  wounds 
(excluding  the  eye)  tended  to  not  be  serious  and  involved  relatively  shallow  soft  tissue  injuries.  Over  half  the 
debris-related  injuries  to  the  face  were  caused  by  Plexiglas  fragments.5  This  type  of  wound  may  become  more 
prevalent  as  dismounted  combat  soldiers  increasingly  use  transparent  eye  and  face  protection. 

The  range  and  extent  of  tissue  response  associated  with  penetrating  facial  wounds  tend  to  be  highly  variable 
and  defy  segregation  into  neat  generic  categories.  Often,  the  missile  trajectory  is  non-linear,  particularly  for 
low-velocity  events.  Tissue  damage  that  occurs  in  conjunction  with  combat-related,  penetrating  facial  wounds 
tends  to  extend  beyond  the  permanent  wound  tract,  tends  to  promote  significant  bone  fracture,  often  with 
ejection  of  secondary  fragments,  and  in  general,  the  resulting  tissue  damage  may  not  be  co-linear  or  even 
contiguous.  The  is  due  to  the  unique  contours  and  complicated  skeletal  geometry  of  the  face,  which  provides 
many  oblique  surfaces  to  deflect  projectiles  and  to  the  tremendous  variation  of  material  properties  in  facial 
structures,  which  tend  to  promote  a  daunting  range  of  tissue  response  during  the  same  event.  This  suggests  that 
trauma  models  that  are  sufficiently  general  to  describe  the  range  and  extent  of  tissue  damage  associated  with 
these  wounds  necessarily  entails  inclusion  of  anatomical  regions  outside  the  paranasal  sinuses  (independently 
of  whether  these  regions  are  displayed  or  available  to  the  end  user).  Presumably,  some  key  trauma  management 
training  issues  are  associated  with  the  extreme  variability  of  these  wounds  and  how  they  are  affected  by 
anthropometric  differences,  nuances  in  projectile  characteristics,  and  variation  in  striking  conditions. 

There  are  several  issues  that  significantly  frustrated  a  general  approach  to  the  first  two  tasks  in  this  effort.  First, 
there  is  no  national  database  of  wound  ballistics  data  and  there  is  no  standard  format  for  classifying  penetrating 
wounds.  Consequently,  the  information  that  exists  is  often  locally  acquired  on  an  informal  basis  and  is  oriented 
to  the  interests  and  medical  specialty  of  the  investigator.  Further,  the  manner  in  which  wound  data  is  aggregated 
significantly  biases  reporting  of  wound  statistics.  Aggregation  of  wound  data  for  statistical  analysis  depends  on 
how  the  human  anatomy  is  divided,  how  wounds  are  categorized,  and  precisely  what  population  is  surveyed. 
Wounds  can  be  categorized  in  terms  of  entry  site,  anatomical  structures  affected,  wound  severity  or  outcome, 
echelon  of  medical  care  surveyed,  sample  size,  theatre  of  operations,  and  causative  agent.  The  resulting 
statistics  can  also  be  biased  by  the  excluded  categories  of  casualties  (e.g.,  intracranial  involvement,  fatalities, 
"carded  for  record  only,"  etc).  Although  this  frustrates  general  approaches  to  the  first  two  tasks,  a  sufficient 
body  of  anecdotal  information  exists  from  which  general  trends  can  nevertheless  be  discerned. 

In  this  effort,  civilian  and  military  trauma  literature  was  exploited  to  gain  insight  in  the  variations  of  tissue 
response  associated  with  penetrating  facial  wounds.  In  general,  an  avulsive  wound  with  massive  tissue  destruc¬ 
tion  of  soft  and  osseous  tissue  characterizes  a  bullet  from  a  high-velocity  military  rifle  striking  the  face.  The 
entrance  wound  is  typically  a  gaping,  devastated  area  of  soft  tissue,  often  with  total  avulsion  of  hard  tissue. 
Low-velocity  handgun  wounds,  however,  present  a  totally  different  spectrum  of  injury  in  the  facial  regions. 
Injuries  are  generally  non-fatal  and  relatively  non-mutilating.  The  entrance  and  exit  wounds  are  both  small  and 
very  often  the  missile  does  not  exit  but  is  retained  within  the  tissue.  Shotgun  injuries  represent  the  entire 
continuum  of  tissue  damage.  At  close  range,  severe  soft  and  hard  tissue  loss,  with  gross  mutilation  of  the 
affected  structures  characterize  shotgun  injuries.  At  long  ranges  however,  shotgun  injuries  are  relatively  non¬ 
mutilating. 


4  Malick,  D.,  “U.S.  Army  Casualties  Aboard  Aircraft  in  the  Republic  of  Vietnam  (1968  through  1970),"  Ballistic 
Research  Laboratory  Contract  Report  257,  August  1975,  p.  97. 

5  Bernier,  R.G.,  and  Smith  H.C.,  “U.S.  Army  Casualties  Aboard  Aircraft  in  the  Republic  of  Vietnam  (1962-1967), 
"Ballistic  Research  Laboratories  Memorandum  Report  No.  2030,  March  1970. 
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A  helpful  way  to  classify  penetrating  facial  injuries  is  to  categorize  them  according  to  whether  the  wound  is 
penetrating,  perforating,  or  avulsive.  In  the  penetrating  wound,  the  missile  is  retained  within  the  injured  tissue. 
This  category  can  be  further  divided  into  "superficial"  and  "deep  penetrating"  injuries  where  the  projectile 
penetrates  deep  fascia  and  may  enter  a  body  cavity.  The  perforating  wound  is  where  the  missile  passes 
completely  through  the  target.  By  definition  there  is  always  an  entrance  and  exit  wound  associated  with 
perforating  wounds.  The  avulsive  wound  is  one  in  which  large  segments  of  hard  and/or  soft  tissue  are 
explosively  ejected  from  the  wound.  Usually  there  is  a  small  entrance  and  gaping  exit  wound.  These  wounds 
have  the  greatest  amount  of  tissue  damage.  Figure  1  shows  how  various  weapons  are  grossly  associated  with  the 
wound  categories  discussed  above. 


Penetrating  Perforating 

Superficial  Deep  fascia,  possibly 

ENTERING  BODY  CAVITY 

Airgun - 

Civilian  Handgun 

-  Low-velocity  fragments/spall - 


Shotgun 

Long  Range  " 

Figure  1.  Gross  association  of  weapons  with  wound  categories 


AVULSIVE 


Military  Rifle 
High-velocity  fragment" 

Close  Range 


Figure  2  shows  an  example  of  a  penetrating  wound  from  an  airgun 
pellet.6  The  airgun  pellet  has  become  flattened  on  impact  with  the 
mandible,  which  is  not  damaged.  Figure  3  shows  another  example  of  a 
penetrating  wound  from  an  artillery  shell  fragment.7  A  Japanese  75-mm. 
artillery  shell  that  exploded  at  a  distance  of  100  yards  hit  a  soldier 
standing  in  the  company  area.  A  fragment  of  the  shell  penetrated  the 
outer  wall  of  the  maxillary  sinus  and  lodged  in  the  sinus  cavity.  This  is  a 
good  example  of  the  relative  protection  afforded  by  bony  structures  to 
low-velocity  fragments.  Figure  4  shows  a  schematic  of  an  avulsive 
wound  from  a  self-inflicted  shotgun  injury.8  The  patient  required  an 
immediate  unilateral  frontal  lobectomy  and  midfacial  debridement. 


Figure  2.  Airgun  pellet  that  has 
become  flattened  on  impact 
with  the  mandible 


6  Walker,  R.V.,  and  Frame,  W.,  “Civilian  maxillo-facial  gunshot  injuries,”  Int.  J.  Oral  Surgery,  13:  253-277, 1984, 
Figure  2,  p.  265. 

7  Oughterson  A.W.,  Hull,  H.C.,  et  al.,  "Study  of  Wound  Ballistics  -  Bougainville  Campaign, "  Chapter  V  in: 
Coates,  J.B.,  and  Beyer,  J.C.,  Wound  Ballistics,  Medical  Department  United  States  Army,  Government  Printing 
Office,  Washington,  D.C.,  December  1962,  Figure  179,  p.  333. 

*  Vasconez,  H.C.,  Shockley,  M.E.,  et  al.,  “High-energy  Gunshot  Wounds,"  Annals  of  Plastic  Surgery,  36(1):  18- 
25,  January  1996,  Figure  2D  and  2E,  p.  22. 
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inflicted  shotgun  injury 

Figure  3.  X-ray  of  skull  showing 
artillery  fragment  lodged  in  maxillary 
sinus 

2  TASK  1  -  FREQUENCY  AND  TYPES  OF  FACIAL  WOUNDS 

This  section  is  divided  into  three  subsections.  This  first  section  reviews  the  various  schemes  of  dividing  the 
facial  anatomy  for  the  purpose  of  aggregating  wounds  for  statistical  reporting.  The  lack  of  commonality  and 
lack  of  spatial  resolution  in  these  various  schemes  complicates  the  task  of  making  inferences  regarding  the 
frequency  of  paranasal  sinus  wounds.  The  second  section  discusses  the  frequency  of  penetrating  wounds  to  the 
face  employing  civilian  trauma  data.  This  data  tends  to  be  more  highly  refined  than  the  corresponding  military 
data  and  is  therefore  helpful  in  discussing  combat-related  penetrating  wounds  to  various  regions  of  the  face  in 
the  third  section. 

2.1  Facial  Delineation 

As  mentioned,  there  is  a  lack  of  commonality  concerning  the 
anatomical  regions  over  which  wound  ballistics  data  is  aggregated. 

Wound  ballistics  data  from  the  military,  in  particular,  tends  to  be 
aggregated  at  a  fairly  low  level  of  spatial  resolution.  This  is  largely 
because  the  data  is  collected  to  design  protective  measures  to 
reduce  casualties,  as  opposed  to  studying  remedial  medical 
procedures.  Figure  5  shows  the  location  of  the  paranasal  sinuses.9 
Figure  6  shows  the  most  common  division  of  the  human  body  by 
the  military  for  wound  ballistic  reporting.10  Beebe  accumulates 
some  military  wound  ballistics  data  in  a  slightly  more  highly 
resolved  format  (see  Figure  7).11  Finally,  Bernier  in  reporting 
wound  data  on  Army  aviators  during  the  Vietnam  conflict  uses  the 
facial  delineation  shown  in  Figure  8.12 


Figure  5.  Anterior  view  of  paranasal 
sinuses 


9  Blakiston's  Illustrated  Pocket  Medical  Dictionary,  Second  Edition,  McGraw-Hill  Book  Company,  Inc.,  New  York, 
NY,  1 960,  Illustrative  palte  11. 

10  Palmer,  A.,  "Survey  of  Battle  Casualties,  Eighth  Air  Force,  June,  July,  and  August  1941,"  Chapter  IX  in  Coates, 
J.B.,  and  Beyer,  J.C.,  Wound  Ballistics,  Medical  Department,  United  States  Army,  Government  Printing  Office, 
Washington,  D.C.,  December  1962,  Figure  275,  p.  560. 

11  Beebe,  G.W.,  and  De  Bakey,  M.E.,  Battle  Casualties,  Incidence,  Mortality,  and  Logistic  Considerations, 
Charles  C.  Thomas  Publisher,  Springfield,  IL,  1952,  p.  88. 
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Figure  6.  Delineation  of  human  anatomy 
by  Palmer 


Figure  7.  Beebe's  topographic 
delineation  of  the  head  and  neck 


12 


Figure  8.  Bernier's  division  of  the  head  and  neck 


Figure  9.  Anatomic  boundaries  as  defined  by  Gussack 


The  delineation  of  the  face  for  reporting  civilian  wound  ballistics  data  tends  to  be  more  highly  resolved  since 
the  data  is  often  used  to  study  treatment  modalities.  The  most  common  division  of  the  face  for  civilian  wound 
ballistic  studies  is  shown  in  Figure  9.13  Dolin's  division  shown  in  Figure  10  tends  to  emphasize  the  bony  as 
structures  of  the  maxillofacial  complex,14  whereas  Kihtir's  delineation  (Figure  11)  includes  the  orbital  region 
well.15 


12  Bernier,  R.G.,  and  Smith  H.C.,  “U.S.  Army  Casualties  Aboard  Aircraft  in  the  Republic  of  Vietnam,  "Ballistic 
Research  Laboratories  Memorandum  Report  No.  2030,  March  1970,  Figure  A-1,  p.  99. 

13  Gussack,  G.S.,  and  Jurkovich,  G.J.,  “Penetrating  Facial  Trauma:  A  Management  Plan.  Southern  Medical 
Journal,”  81  (3):  297-302, 1988,  Figure  1 ,  p.  298. 

14  Dolin,  J.,  Scalea,  T.,  et  al.,  “The  Management  of  Gunshot  Wounds  to  the  Fac e,”  Journal  of  Trauma.  33(4): 
508-515, 1992,  Figure  4,  p.  513. 

15  Kihtir,  R.  Ivatury,  R.R.,  et.a!„  “Early  Management  of  Civilian  Gunshot  Wounds  to  the  Face,”  Journal  of 
Trauma,  35(4):  569-577, 1993,  Figure  1,  p.  572. 
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Figure  11.  Kihtir's  Facial  Delineation 


2.2  Penetrating  wounds  to  the  face  from  civilian  data 

Twelve  retrospective  studies  concerning  penetrating  wounds  to  the  face  from  civilian  data  are  reviewed  below. 

2.2.1  International  data16 

Becelli,  using  data  garnered  from  undocumented  sources,  classified  maxillofacial  wounds  based  on  entrance 
location.  He  divided  the  face  into  thirds,  where  the  upper  third  was  above  the  above-roof  of  orbit,  the  middle 
third  was  between  roof  of  orbit  and  upper  lip,  and  the  lower  third  was  between  lower  lip  and  the  hyoid  bone. 
Among  bony  tissue  fractures  in  the  maxillofacial  region: 

•  42%  involved  the  upper  third 

•  28%  concern  middle  third 

•  3%  concern  lower  third 

•  27%  involve  all  three  sites 

2.2.2  Grady  Memorial  Hospital  (Atlanta)  from  1962-197217 

Bostwick  retrospectively  studied  547  consecutive  cases  of  cervical  and  facial  penetrating  injuries,  277  face  and 
270  neck,  at  the  Grady  Memorial  Hospital  in  Atlanta  from  1962-1972.  Causative  agents  for  the  wounds  were 
373  pistol,  1 1 1  knife  and  razor,  47  shotgun,  and  16  from  miscellaneous  causes  (knitting  needles,  biological 
dissection  instruments).  Specific  injuries  from  Bostwick's  study  are  shown  in  Table  1. 

2.2.3  South  Africa 

Cohen  and  Shakenovsky  retrospectively  analyzed  40  cases  of  low-velocity  handgun  injuries  of  the  face  and  jaw 
from  the  Johannesburg  Group  of  teaching  hospitals.18  All  cases  in  which  the  missile  had  penetrated  the  cranial 
cavity  causing  death  were  excluded.  Other  cases  excluded  were  those  in  which  the  patient  died  from  other  or 
related  injuries. 


16  Becelli,  R.,  De  Ponte  F.S.,  et  al.,  “Firearm  Injuries  in  Maxillofacial  Region  Reconstructive  Surgery,”  Journal  of 
Craniofacial  Surgery,  6(6):  473-476,  November  1995. 

17  Bostwick  J.,  Schneider,  W.J.,  et  al.  “Penetrating  Injuries  of  the  Face  and  Neck,”  Southern  Medical  Journal, 
76(69):  550-553,  May  1976. 

18  Cohen,  M.  A.,  Shakenovsky,  B.N.,  etal.,  “Low  Velocity  Hand-Gun  Injuries  of  the  Maxillofacial  Region,”  Journal 
of  Maxillofacial  Surgery,  14:26-33, 1986. 
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The  majority  of  cases  were  due  to  assault  (49%),  Table  1.  Specific  Injuries  from  Bostwick 
followed  by  accidental  shootings  (27%)  and 
failed  suicide  attempts  (24%).  Injuries  ranged 
from  mild  to  soft  tissue  damage  without  fractures 
to  severe,  comminuted  fractures  of  the  facial 
bones  with  severe  soft  tissue  damage.  Rarely 
were  injuries  life-threatening.  Entrance  wounds 
were  characteristically  small  and  well  circum¬ 
scribed.  In  60%  of  the  cases,  the  bullet  did  not 
exit  and  was  retained  within  the  tissues. 

Injuries  were  divided  into  upper  third,  middle 
third,  and  lower  third  of  the  face.  Upper-third 
injuries  were  above  the  supraorbital  ridges. 

Middle-third  injuries  occurred  in  the  area  from 
the  supraorbital  ridges  to  the  upper  lip,  and 
lower-  third  facial  injuries  involved  the  area 
between  the  lower  lip  and  hyoid  bone. 

In  25  cases,  the  caliber  of  the  bullet  was  known, 

.38  in  52%  followed  by  .22  in  24%.  The  authors  observed  that  the  most  severe  injuries  were  due  to  the  .22 
caliber  projectile.  The  .22  most  likely  has  a  lower  mass  and  striking  velocity  than  the  .38  and  therefore  less 
kinetic  energy.  According  to  conventional  wisdom  (with  which  the  author  does  not  agree)  which  correlates 
wound  severity  with  kinetic  energy,  we  would  expect  more  tissue  damage  to  be  associated  with  the  .38. 
However,  .22  handguns  tend  to  use  a  full  metal  jacket  (FMJ)  projectile  as  opposed  to  the  .38,  which  uses  a 
relatively  soft  projectile.  We  may  speculate  that  the  .38  bullet  probably  tends  to  deform  when  it  strikes  bone 
tissue  as  opposed  to  the  .22. 

Most  entrance  wound  sites  were  located  around  the  mouth  and  chin  (25%),  followed  by  cheek  (20%)  and  ear 
(15%).  The  cheek  was  the  most  common  exit  site,  followed  by  the  neck  area. 

Mandibular  fractures  varied  from  simple  fractures  to  severely  comminuted  compound  fractures  with 
displacement  of  bony  fragments.  Of  23  mandibular  fractures,  18  (78%)  were  comminuted  fractures,  while  5 
were  not.  The  maxillary  complex  was  involved  in  45%  of  the  cases,  of  which  50%  involved  a  fracture 
encompassing  the  sinuses.  Ninety  percent  of  the  wounds  involved  single  or  multiple  fractures  of  the  facial 
skeleton.  The  mandible  was  the  bone  most  often  fractured  (23  cases)  followed  by  the  maxillary  complex  (18 
cases).  Corresponding  to  this,  fractures  of  the  lower  third  of  the  face  occurred  in  the  majority  of  cases,  followed 
by  concomitant  fractures  of  the  middle  and  lower  thirds  of  the  facial  skeleton. 

Fractures  of  the  mandible  were  comminuted  in  the  majority  of  cases,  with  the  extent  of  damage  varying.  In 
some  instances,  the  bullet  shattered  the  entire  body  or  condyle  of  the  mandible.  The  body  of  the  mandible  was 
the  site  most  commonly  affected  (52%)  with  symphyseal,  ramus,  and  conylar  fractures  the  next  most  common 
sites,  occurring  with  equal  frequency  (12%).  Combination  fractures,  such  as  body  fracture  combined  with 
dentoavelor  fracture,  were  present  in  many  cases.  In  comminuted  fractures,  bony  fragments  and  pieces  of  tooth 
act  as  secondary  missiles,  causing  further  soft  tissue  damage. 

Maxillary  complex  fractures  were  generally  less  severe  than  mandibular  fractures.  This  is  probably  due  to  the 
relatively  thin  nature  of  the  bone,  which  allows  the  bullet  to  pass  through  rather  than  stopping  it.  When  the 
bullet  passed  through  the  mouth  into  the  maxillary  sinuses,  an  oroantral  fistula  resulted.  In  several  cases,  these 


Globe 

108 

Fracture  of  maxillary  bone  and  sinus 

64 

Mandibular  fractures 

63 

Pharynx 

35 

Zygoma  fractures 

26 

Orbital  fractures 

26 

Tongue 

19 

Facial  nerve 

11 

Frontal  sinus 

10 

Paratoid 

10 

Nasal  fractures 

10 

Sphenoid  sinus 

7 

Facial  vein 

5 

Facial  artery 

4 

Palate 

3 

Ethmoid  sinus 

2 

Homer's  syndrome 

1 
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were  fairly  large.  Similarly,  several  oronasal  communications  resulted.  The  tongue  and  roof  of  the  mouth  were 
the  most  common  sites  of  soft  tissue  injury.  These  injuries  rarely  threaten  the  airway. 

Cohen  and  Byes-Varley19  reviewed  27  cases  from  South  Africa  where  bullets  penetrated  and  were  retained  in 
the  maxillofacial  region.  Two  involved  the  maxillary  sinus. 

2.2.4  Wake  Forest  University  Medical  Center,  1986-199220 

Forty  cases  of  gunshot  wounds  to  the  mouth,  mandible,  and  maxilla  were  treated  at  Wake  Forest  University 
Medical  Center  over  7  years,  1986-1992.  The  study  conducted  by  Cole 
excluded  patients  with  gunshot  wounds  involving  only  the  neck  or 
with  isolated  intracranial  injuries.  In  Cole's  study,  67.5%  of  the  cases 
were  due  to  assault,  17.5%  accidental  shooting,  and  15%  unsuccessful 
suicide.  The  causes  of  injury  in  3 1  cases  were  handguns,  8  involved 
shotguns,  and  1  involved  a  high-velocity  rifle.  Of  the  40  wounds:  22 
were  midfacial  entrance  wounds,  1 1  involved  isolated  mandibular 
entrance  wounds,  and  7  were  combined  mandibular  and  midfacial 
entrance  wounds.  Figure  12  shows  the  distribution  of  entrance 
wounds  from  Cole's  study. 

2.2.5  SUNY  Health  Center,  Brooklyn,  July  1986  to  April  199121 

From  1  July  1986  to  1  April  1991,  Dolin  reviewed  the  cases  of  100 
patients  with  gunshot  wounds  to  the  face  (98  gunshot,  2  shotgun) 
which  were  seen  at  the  SUNY  Health  Center  in  Brooklyn.  Note  that 
gunshot  wounds  to  the  face  are  relatively  rare,  even  in  busy  urban 
centers  such  as  SUNY  Brooklyn.  In  fact,  in  a  discussion  comment  Figure  u  Distribution  of  entrance 
following  Dr.  Dolin’s  paper.  Dr.  Rao  Ivatury  indicated  that  he  had  seen  woun(js  from  Cole's  study 
only  39  cases  of  gunshot  wounds  to  the  face  over  four  years  in  the 
Bronx. 

Dolin  defined  a  penetrating  wound  of  the  face  as  one  where  the  entrance  wound  was  located  above  the  inferior 
border  of  the  mandible  and  below  the  base  of  the  skull.  Patients  whose  entrance  wounds  were  located  above  the 
superior  bridge  of  the  nose  or  orbital  roof  were  considered  to  have  transcranial  wounds  (Figure  10).  Patients 
whose  bullets  entered  in  zones  II  and  II  of  the  neck  were  excluded.  There  were  10  cases  of  multiple  gunshot 
wounds,  four  to  the  face.  In  Dolin's  population: 

•  35  patients  needed  urgent  airway  control  in  the  Emergency  Department 

•  2  patients  needed  surgical  airway  control 

•  19  sustained  vascular  injuries,  1 1  of  which  required  therapy 

•  9  suffered  intracranial  wounds 

•  67  sustained  bony  injuries  (110  wounds) 

•  38  suffered  significant  neurologic  injury  (26  peripheral,  only  10  of  which  were  obvious  at  the  time  of 
initial  evaluation,  one  spinal  and  20  cerebral) 

•  6  died  (3  CNS,  one  exsanguination,  and  2  sepsis) 

19  Cohen,  M.A.,  and  Boyes-Varley,  G.,  “Penetrating  Injuries  to  the  Maxillofacial  Region,”  J  Oral  Maxillofac  Surg. 
44:197-202,  1986. 

20  Cole,  R.D.,  Browne,  J.D.,  et  al.,  “Gunshot  wounds  to  the  mandible  and  midface:  Evaluation,  treatments,  and 
avoidance  of  complications,”  Otolaryngology-  Head  and  Neck  Surgery,  111(6):  739-745,  December  1974. 

21  Dolin,  J.,  Scalea,  T.,  et  al.,  “The  Management  of  Gunshot  Wounds  to  the  Fac e,”  Journal  of  Trauma.  33(4): 
508-515,  1992. 
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Maxillary  fractures  involving  the  sinuses  were  the  most  common  injury  (33),  followed  by  mandibular  fracture 
(29).  Closed  reduction  of  facial  fractures  was  the  most  common  of  the  59  procedures  performed;  only  3  patients 
underwent  an  open  operation  for  facial  fractures.  The  facial  nerve  was  the  most  common  peripheral  nerve 
injured,  followed  by  mandibular  branch  of  the  trigeminal  nerve.  Forty-four  percent  of  Dolin's  cases  had  surgical 
treatment,  of  which  25%  had  post-surgical  complications.  Results  from  Dolin's  study  for  different  regions  of  the 
face  are  shown  in  Table  2. 

Table  2.  Results  from  Dolin's  Study 


(See  Figure  6  for  definition  of  facial  zones) 


Facial  Zone 

#  patients 

ED  Airway 

CNS 

Vascular 

Peripheral 

A 

42 

9  (21%) 

7  (17%) 

6  (14%) 

12  (29%) 

B 

33 

12  (36%) 

7  (21%) 

4  (12%) 

6  (18%) 

C 

42 

20  (48%) 

3  (7%) 

9  (21%) 

11  (26%) 

CNS  injuries  are  a  common  complication  of  transfacial  gunshot  wounds.  The  most  interesting  aspect,  however, 
involved  the  patients  who  had  missile  trajectories  that  traversed  the  base  of  the  skull.  Although  initially 
asymptomatic,  Dolin  found  that  9  of  15  patients  (60%)  had  significant  intracranial  injuries. 

2.2.6  San  Francisco  General  Hospital,  1971  and  197822 

Gant  and  Epstein  evaluated  66  cases  of  low-velocity  gunshot  wounds  to  the  maxillofacial  complex  between 
1971  and  1978  at  San  Francisco  General  Hospital.  Patients  dead  on  arrival  (DOA)  or  who  died  within  24  hours 
were  excluded  from  this  study.  The  cases  were  divided  into  three  anatomic  regions,  based  on  the  wound 
entrance  location: 

•  Area  I:  Supraorbital  (36  cases) 

•  Area  II:  Midface  between  the  level  of  the  supraorbital  ridges  to  upper  lip  (20  injuries,  12  cases  of  soft 
tissue  injury  only) 

•  Area  ID:  Lower  lip  to  the  hyoid  (10  cases) 

Wounds  in  Gant  and  Epstein's  population  are  summarized  in  Table  3.  In  their  population,  59  of  66  cases  were 
found  to  have  an  entrance  wound  only,  with  the  missile  expending  itself  in  the  soft  or  bony  tissues. 


Table  3.  Wounds  in  Grant  and  Epstein's  Population 


Supraorbital,  36  cases 

Mid-facial.  20  cases 

Lower  facial,  10  cases 

Entered  cranial  cavity 

20 

Soft  tissue  only 

12 

Facial  fractures 

6 

CNS 

13 

Facial  fractures 

8 

Soft  tissue 

4 

Soft  tissue  only 

4 

Periorbital  fractures 

6 

Carotid  laceration 

3 

Ear  or  mastoid 

3 

Teeth  fractures 

2 

Tracheostomy 

2 

Cranial  nerves 

3 

Cranial  nerves 

2 

Infection 

1 

Sinuses 

3 

Enucleation 

2 

Direct  ocular 

3 

Infections 

0 

Infection 

2 

22  Gant,  T.D.,  and  Epstein,  L.I.,  “Low-velocity  Gunshot  Wounds  to  the  Maxillofacial  Complex,”  Journal  of  Trauma. 
19(9):  674-77, 1979. 
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2.2.7  University  of  South  Alabama  Trauma  Center,  June  1985  to  June  198623 

Gussack  and  Jurkovich  evaluated  16  cases  of  penetrating  facial  trauma  treated  at  University  of  South  Alabama 
Trauma  Center  from  June  1985  to  June  1986.  The  cases  included  13  gunshot  and  3  stab  wounds.  Gunshot 
wounds  included  10  low-velocity  handguns,  one  .357  magnum,  one  pellet  rifle,  and  one  12-gauge  shotgun. 
Injuries  included  3  CNS,  4  mandibular  fractures,  3  optic  nerve  or  globe  injuries,  and  7  maxillary  sinus  fractures. 

In  evaluating  patient  wounds,  Gussack  and  Jurkovich  divided  the  face  into  three  regions  (see  Figure  5).  Area  I 
includes  the  forehead  and  ears,  and  is  bounded  superiorly  by  the  hairline  and  inferiorly  by  the  supraorbital  rims. 
Area  II  is  the  midportion  of  the  face,  including  the  area  from  the  supraorbital  ridges  to  the  upper  lip,  and 
extending  to  the  preauricular  areas  laterally.  Area  IE  extends  from  the  lower  lip  to  the  hyoid  bone.  Of  16 
patients  reviewed,  the  entrance  wound  was  isolated  to  Area  I  in  three  patients,  to  Area  II  in  nine  and  to  Area  HI 
in  three.  One  patient  had  a  gunshot  to  Areas  I  and  II.  The  ten  Area  II  wounds  resulted  in  intracranial  injury  in 
one  patient,  optic  nerve  or  globe  injury  in  three,  and  maxillary  sinus  injury  in  seven.  Three  patients  sustained 
soft  tissue  damage  only. 

2.2.8  Libya24 

Khalil  reviewed  18  patients  with  different  types  of  what  he  classifies  as  “civilian”  gunshot  injuries  to  the  face 
and  jaw.  The  causative  agents  included  5  cases  from  rifles,  2  from  handguns,  4  from  shotguns,  5  from  airguns, 
and  2  cases  of  shrapnel  from  land  mines.  In  this  population,  10  patients  had  soft  tissue  injuries  and  8  had 
mandibular  fractures  with  soft  tissue  injuries. 

2.2.9  Lincoln  Medical  and  Mental  Health  Center  (Bronx),  1988-199225 

Kihtir  reviewed  54  civilian  patients  from  1988  to  1992  with  gunshot  (handgun)  wounds  of  the  face  treated  at 
the  Lincoln  Medical  and  Mental  Health  Center  in  the  Bronx.  Suicide  and  shotgun  injuries  were  excluded  from 
this  review  as  well  as  patients  with  entrance  wounds  in  the  frontal  region.  In  Kihtir's  population: 

•  2  patients  (4%)  sustained  multiple  gunshot  wounds 

•  26  patients  (48%)  sustained  single  facial  fractures 

•  17  (3 1%)  had  multiple  facial  fractures 

•  11  patients  (20%)  had  soft-tissue  injury  without 
concomitant  fracture 

•  18  (33%)  required  urgent  airway  control  and  39%  non¬ 
emergency  airway  control 

•  12  (22%)  sustained  CNS  injuries 

•  5  (9%)  had  vascular  injuries 

Kihtir  divided  the  face  into  the  mandible  and  tongue  (lower  face); 
maxilla,  lower  portion  of  nose  and  zygoma  (mid-face);  and  orbits 
and  nasoethmoid  complex  (orbital).  See  Figure  7.  The  maxilla 
was  the  most  commonly  fractured  bone  (41%),  followed  by 
mandible  (28%).  Figure  13  shows  the  proportion  of  bone 
fractures  in  Kihtir's  population.26 


ORBITAL  9  (17%) 

NASAL  7  (13%) 
ZYGOMA  7  (13%) 
MAXILLA  22  (41%) 
PALATE  5  (9%) 


MANDIBLE  IS  (28%) 


NONE  11  (20%) 


Figure  13.  Proportion  of  bone  fractures 
in  Kihtir's  population 


23  Gussack,  G.S.,  and  Jurkovich,  G.J.,  “Penetrating  Facial  Trauma:  A  Management  Plan.  Southern  Medical 
Journal ,”  81  (3):  297-302, 1 988. 

24  Khalil,  A.F.,  “Civilian  Gunshot  Injuries  to  the  Face  and  Jaws,”  British  Journal  of  Oral  Surg.  18:205-21 1, 1980. 

25  Kihtir,  R.  Ivatury,  R.R.,  et  at.,  “Early  Management  of  Civilian  Gunshot  Wounds  to  the  Face,”  Journal  of 
Trauma,  35(4):  569-577,  1993. 

26  Ibid.,  Figure  2,  p.  572. 
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2.2.10  King’s  County  Hospital  (Brooklyn),  1985  -  199427 

Lee  retrospectively  reviewed  35  patients  treated  for  low-velocity  gunshot  wounds  of  the  paranasal  sinuses 
treated  at  King’s  County  Hospital  in  Brooklyn  between  1985  and  1994.  All  patients  with  isolated  intracranial, 
neck  (including  zone  HI),  orbital,  or  mandibular  injuries  were  excluded  from  this  review.  The  total  number  of 
sinuses  was  greater  than  35  because  many  patients  had  more  than  one  sinus  injury.  The  maxillary  sinus  was 
most  frequently  injured,  followed  by  ethmoid,  frontal,  and  sphenoid.  Fifteen  patients  had  peripheral  nerve 
injuries  including  the  maxillary  branch  of  trigeminal  nerve  (7),  followed  by  facial  nerve  (6),  and  mandibular 

branch  of  trigeminal  nerve  (5).  There 

Table  4.  Location  of  bullet  entrance  wound  vs  location  of  injury  were  3  deaths,  each  involved  the 

frontal  sinus  associated  with 
intracranial  injury.  Generally,  the 
wounds  were  nonfatal  and  non¬ 
mutilating.  Table  4  compares  the 
location  of  the  bullet  entrance  wound 
with  the  location  of  injury  from  Lee's 
data. 


Location  of  bullet  entrance 


Location  of  injury 


Frontal  sinus 

6 

Ethmoid 

20 

Orbital 

4 

Maxillary 

27 

Maxillary 

20 

Frontal 

6 

Nasal 

5_ 

Sphenoid 

5 

Total 

35 

2.2.1 1  Mount  Sinai  School  of  Medicine  (Manhattan),  January  1963  through  December  196628 

Yao  reviewed  data  from  all  patients  (60  cases)  who  sustained  missile  wounds  of  the  face  from  bullet  wounds  or 
shotgun  blasts  during  the  period  from  January  1963  through  December  1966  who  were  treated  at  Mount  Sinai 
School  of  Medicine  in  New  York 
City.  Wounds  of  the  forehead  and 
brain  were  excluded.  Tissue 
injuries  were  classified  into  those 
that  damaged  bony  structures  and 
soft  tissues.  See  Table  5.  Two 
patients  sustained  facial  nerve 
injury;  one  of  these  lesions  was 
transient  and  one  was  permanent. 


Table  5.  Damage  to  soft  tissue  and  bony  structures  in  Yao  s  study 


Bony  Structures 

Soft  Tissues 

Mandible 

25 

Skin 

60 

Maxilla 

28 

Tongue 

14 

Zygoma 

3 

Pharynx 

12 

Cervical  Spine 

6 

Carotid  Artery 

4 

Palate 

3 

Eye 

2 

Nasal  Bone 

4 

Sinus 

8 

Orbit 

5 

Salivary  Gland 

5 

Spinal  Cord 

1 

2.2.12  Kentucky  Chandler  Medical  Center  between  1976  and  199329 

During  the  1 8-year  period  from  1976  to  1993,  a  total  of  604  gunshot  wounds  to  the  head  and  neck  were  treated 
at  the  University  of  Kentucky  Chandler  Medical  Center  or  the  Veterans  Administration  Medical  Center.  Of 
these,  33  patients  (5.4%)  were  determined  to  have  gunshot  wounds  to  the  face. 

Of  the  total  number  of  injuries,  73%  (24  cases)  were  caused  by  shotguns.  These  weapons  were  almost  entirely 
12-gauge  in  caliber.  Handguns  were  used  in  7  cases,  5  of  which  were  either  assaults  or  accidental  shootings. 
These  weapons  included  four  .38  caliber  pistols,  two  .357  magnum  handguns,  and  one  9-mm  pistol.  Only  1 
patient  was  known  to  have  been  shot  from  a  range  of  more  than  10  feet.  For  1  patient,  the  range  was  unknown; 
for  the  remaining  30,  the  range  was  10  feet  or  less. 


27  Lee,  D.,  Nash,  M.,  et  al.,  “Low-velocity  gunshot  wounds  to  the  paranasal  sinuses,”  Otolaryngology-Head  and 
Neck  Surgery,  116(3):  372-78, 1997. 

28  Yao,  S.T.,  Vanecko,  R.M.,  etal.,  “Gunshot  wounds  of  the  face,”  Journal  of  Trauma,  12(6):  523-528, 1972. 

29  Vasconez,  H.C.,  Shockley,  M.E.,  et  al.,  “High-energy  Gunshot  Wounds,"  Annals  of  Plastic  Surgery,  36(1):  18- 
25,  January  1996. 
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Injuries  were  categorized  by  facial  region  into  upper  (skull  and  intracranial),  middle  (maxilla  and  maso- 
ethmoid-orbital),  and  lower  (mandible  and  submental).  Every  injury  involved  both  bone  and  soft  tissue,  and 
more  than  half  (17)  involved  multiple  facial  regions. 

Thirteen  patients  sustained  permanent  neurological 
sequelae,  either  sensory  or  motor.  Table  6  shows  the 
distribution  of  facial  wounds  according  to  anatomic 
involvement. 


More  than  90%  of  the  patients  in  Yao's  study 
sustained  injuries  to  the  lower  face  and  mandible, 
which  is  a  typical  pattern  of  injury  in  self-inflicted 
wounds  with  long-bore  firearms  such  as  shotguns. 


Table  6.  Distribution  of  facial  wounds 
according  to  anatomic  involvement 


TvDe  of  Injury 

Upper 

8 

24.2% 

Midface 

22 

66.7 

Lower 

25 

75.8 

Upper  only 

1 

3.0 

Midface  only 

4 

12.1 

Lower  only 

10 

30.3 

Upper-midface 

3 

9.1 

Lower-  midface 

11 

33.3 

Upper-mid-lower 

4 

12.1 

2.3  Combat-related,  penetrating  wounds  to  the  face 
This  section  is  divided  into  four  subsections: 

•  2.3.1  World  War  II  and  the  Korean  War  -  discusses  the  significance  of  facial  wounds  in  WWII  and 
the  Korean  War 

•  2.3.2  Vietnam  Conflict  -  discusses  this  issue  in  the  context  of  the  Vietnam  conflict 

•  2.3.3  Regional  Conflicts  -  discusses  this  issue  in  the  context  of  limited  sample  sizes  from  the  Afghan 
and  Lebanon  Wars 

•  2.3.4  U.S.  Army  Aviators  -  discusses  the  occurrence  of  facial  wounds  in  WWII  and  Vietnam-era  U.S. 
Army  aviators. 

2.3.1  World  War  II  and  the  Korean  War 

Beebe  estimates  that  head,  neck,  and  facial  wounds  represented  21%  of  total  casualties  during  WWII.30  Of 
461,258  living  wounded  during  WWII,  Beebe  also  estimates  that  14-15%  were  head,  neck,  and  facial  wounds.31 
For  a  sample  of  the  7,714  wounds  to  head,  neck,  and  face,  the  rate  of  facial  wounds  was  39.7%.  (Figure  7 
shows  how  Beebe  delineated  anatomical  regions  of  the  face.)  From  the  front,  the  face  extended  from  the  point 
of  the  chin  to  the  eyebrows  and  supra-orbital  ridges.  In  side  view,  the  face  was  demarcated  from  the  neck  by  a 
line  from  external  auditory  meatus  to  the  angle  of  the  mandible  and  thence  along  the  lower  border  of  the 
mandible  to  the  chin.  Tables  7,  8  and  9  elaborate  on  the  distribution  of  wounds  for  different  theatres  of 
operation  by  facial  region  and  broad  anatomical  structure  during  WWII.  Table  10  gives  comparative 
information  for  WWII  and  the  Korean  War  on  the  occurrence  of  head,  neck,  and  facial  wounds. 


30  Beebe,  G.W.,  and  De  Bakey,  M.E.,  Battle  Casualties,  Incidence,  Mortality,  and  Logistic  Considerations, 
Charles  C.  Thomas  Publisher,  Springfield,  Illinois,  1952,  Table  78. 

31  Ibid.,  Table  81,  p.  181. 
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Table  7.  Case  fatality  rate  of  wounded  (admissions  to  hospital  and  quarters)  in  Southwest  Pacific 
Area,  1942-1943  by  broad  anatomical  region  and  structures34 


#  Wounded 

#  Deaths 

%  Deaths 

FACE 

126 

7 

5.6 

Soft  tissues  only 

96 

0 

0.0 

Soft  tissues  and  bones 

30 

7 

23.3 

TOTAL 

2,602 

145 

5.57 

Table  8.  Case  fatality  rates  for  wounds  (excluding  living  who  lost  no  duty  time)  of  facial  region  by 
tissues  involved,  all  wounded  in  SWPA,  MTO,  and  POA,  January-June  194433 


Organs  and  Tissues 

#  Wounded 

#  Deaths 

%  Deaths 

FACE 

3,062 

54 

2 

Eye 

798 

4 

1 

Nose 

148 

4 

3 

Mouth  and  other  organs 

158 

2 

1 

Bones 

407 

19 

5 

Joints  and  muscles 

7 

0 

0 

*  Of  7,714  wounds  to  the  head,  neck,  and  face,  the  face  represented  39.7%. 


Table  9.  Distribution  of 20,747  American  battle  casualties  admitted  to  Fifth  U.S.  Army 
hospitals,  1  August  1944-2  May  1945,  by  principal  wound  in  the  head  and  neck32 


%  of  all  Battle 

Casualties 

%  Died 

Head 

Intracranial 

1.95 

26.00 

Scalp 

3.775 

0 

Eye  and  Ear 

2.00 

0 

Neck 

2.07 

0.87 

Maxillofacial 

Bone 

1.22 

1.5 

Soft  tissue 

4.68 

0.29 

32  Snyder,  H.E.,  and  Culbertson,  J.W.,  "Study  of  Fifth  U.S.  Army  Hospital  Battle  Casualty  Death,"  Chapter  VII  in 
Coates,  J.B.,  and  Beyer,  J.C.,  Wound  Ballistics,  Medical  Department,  United  States  Army,  Government  Printing 
Office,  Washington,  D.C.,  December  1 962,  Table  128,  p.  482. 

33  Ibid.,  Table  40,  p.  112. 

34  Ibid.,  Table  37,  p.  108. 
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Table  10.  Percent  distribution  for  anatomical  location  of  wound  (excludes  cases  where  specific 
location  was  not  recorded)  by  casualty  category36 


Deaths 

WWII 

KIA 

DOW 

Nonfatal 

Deaths 

Korean  War 
KIA  DOW 

Nonfatal 

Head 

32.5 

35.6 

10.7 

7.4 

35.7 

37.9 

25.4 

6.8 

Face 

3.7 

3.8 

3.2 

7.6 

5.0 

4.9 

5.4 

9.4 

Neck 

4.2 

4.6 

2.2 

1.6 

4.5 

5.1 

1.4 

2.0 

Of  91  patients  returned  to  duty  from  the  first  echelon  during  the  Bougainville  campaign  during  WWII,  there 
were  293  soft-tissue  wounds,  20  of  which  were  to  the  face  and  neck.  From  the  rear  echelon,  82  patients  were 
returned  to  duty  with  186  soft  tissue  wounds,  of  which  17  were  face  and  neck  (excluding  the  eye).  There  were  a 
total  of  151  soft-tissue  wounds  and  fractures  in  58  patients  evacuated  to  the  U.S.,  of  which  the  face  and  neck 
accounted  for  12  wounds.35 

Of  1,105  consecutive  casualties  with  penetrating  craniocerebral  trauma  suffered  in  the  Korean  War  during  the 
period  1  September  1950  to  31  August  1952,  43  (3.9%)  were  found  to  have  facio-orbito-cranial  wounds  (4). 
Seventeen  of  these  cases  (40%)  had  involvement  of  the  paranasal  sinuses.37 

2.3.2  Vietnam  Conflict 

Hardaway  estimates  that  approximately  24%  of  those  wounded  in  Vietnam  had  head  and  neck  wounds.38 
Kovaric,  reviewing  3,954  Vietnam-era  patient  records  encompassing  5,577  wounds  occurring  between 
February-November  1967,  estimated  that  15.7%  were  head  and  neck  wounds.39  Based  on  750  patients  treated  at 
the  2nd  Surgical  Hospital  located  in  the  central  highlands  of  the  Republic  of  Vietnam,  over  the  nine-month 
period  from  January  to  October  1966,  Rich  estimated  that  of  1,178  wounds,  123  (10%)  occurred  in  the  head  and 
neck  regions.40  In  comparison  to  Rich's  analysis  of  casualties,  which  were  mostly  survivors,  Jones41  and  Felitis42 
in  studying  casualties  admitted  to  hospitals,  indicated  lower  percentages  of  head  and  neck  wounds,  8.2%  and 
1 1 .3%,  respectively.  According  to  a  WDMET43  study  encompassing  925  wounds,  the  body  region  most 
frequently  sustaining  penetrating  wounds  was  the  head  and  neck  with  260  wounds  (28.10%).  These 
discrepancies  in  the  rates  of  head,  neck,  facial  wounds  probably  reflect  biases  in  the  casualty  population 
surveyed  (echelon  of  medical  care,  wound  severity,  and  predominate  small-arm  threats  in  the  locality)  as 
opposed  to  real  trends.  Table  1 1  compares  the  percent  of  fatal  and  non-fatal  wounds  for  the  head,  neck  and  face 
during  WWII,  the  Korean  War,  and  the  Vietnam  conflict  using  large  sample  sizes  over  long  time  intervals. 


35  Oughterson  A.W.,  Hull,  H.C.,  et  al.,  "Study  of  Wound  Ballisitcs  -  Bougainville  Campaign,"  Chapter  V  in  Coates, 
J.B.,  and  Beyer,  J.C.,  Wound  Ballistics,  Medical  Department,  United  States  Army,  Government  Printing  Office, 
Washington,  D.C.,  December  1962,  Figure  179,  p.  331. 

36  Reister,  F.A,  “Battle  Casualties  and  Medical  Statistics  -  U.S.  Army  Experience  in  the  Korean  War,”  Surgeon 
General,  Department  of  the  Army,  Government  Printing  Office,  1973,  Table  45. 

37  Dillon,  J.D.,  and  Meirowsky,  A.M.,  “Facio-Orbito-Cranial  Missile  Wounds,”  Surg.  Neurol. ,4:  515-518, 1975. 

38  Hardaway,  R.M.,  “Vietnam  Wound  Analysis,”  Journal  of  Trauma,  18(9):  635-642, 1978. 

39  Kovaric,  J.J.,  Aaby,  G.,  et  al.,  “Vietnam  Casualty  Statistics,”  Arch  Surg.  98: 150-52, 1967. 

40  Rich,  N.M.,  “Vietnam  Missile  Wounds  Evaluated  in  750  Patients,”  Military  Medicine,  9-22, 1968. 

41  Jones,  E.L.,  Peters,  A.F.  and  Gasior,  R.M.,  "Early  Management  of  Battle  Casualties  in  Vietnam,"  Arch  Surg, 

97, 1-15. 

42  Felitis,  J.M.,  Jr.,  "Surgical  Experiences  in  Combat  Zone,"  Amer  J  Surg.,  119,  275-278, 1970. 

43  “Evaluation  of  Wound  Data  and  Munitions  Effectiveness  in  Vietnam,  Volume  I”  Final  Report  for  the  Joint 
Technical  Coordinating  Group  for  Munitions  Effectiveness,  December  1970,  Table  III. 
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The  head  and  neck  comprise 

Table  11.  Percent  fatal  and  non-fatal  wounds  to  the  head,  neck,  and  face44  between  9  and  12%  of  total 


War 

Sample  Size 

Head,  Face,  and  Neck 

““  body  projected  surface  area. 
Assuming  non-aimed  fire  and 

WWII 

116,649 

25% 

random  hits,  one  would 

Korea 

90,841 

19 

predict  that  those  areas  should 

Vietnam 

23,396 

15 

_  receive  9  to  12%  of  all  hits. 

However,  Table  1 1  shows 


that  the  head  and  neck  have  generally  received  a  disproportionately  high  number  of  hits.  This  is  undoubtedly 
because  the  head  is  preferentially  exposed  in  combat  as  the  soldier  constantly  monitors  his  environment  by 
means  of  his  eyes,  ears,  and  nose  to  enhance  his  chances  of  survival. 


Table  12  seems  to  indicate  that  the  incidence  and  severity  of  neck  and  facial  wounds  is  increasing.  Rich 
estimates  that  missile  wounds  of  the  maxillofacial  region  accounted  for  approximately  15%  of  all  wounds  in  the 
Vietnam  conflict.  Of  these  combat  wounds,  38%  were  gunshot  wounds,  52%  were  fragment  wounds,  and  about 
10%  were  other  types. 45  This  overall  rate  of  maxillofacial  wounds  estimated  by  Rich  is  greater  than  wounds  to 
the  thorax  and  abdomen  combined.  This  however 
is  probably  an  artifact  of  the  fact  that  body  armor 
worn  during  Vietnam  which  would  reduce  the 
occurrence  and  severity  of  thoracic  and  abdominal 
wounds  from  fragments. 

In  a  WDMET46  analysis  of  the  regional  distribution 
of  fatalities  from  500  consecutive  autopsies  (July 
1967  -  November  1968),  it  was  estimated  that  12 
wounds  (2.4%)  were  facial.  Bellamy  estimates  of 
all  facial  injuries  in  the  WDMET  database,  30% 
involved  the  eye  (one-half  resulted  in  loss  of  the 
eye),  30%  were  injuries  involving  the  jaw,  with 
mandibular  fractures  being  most  common;  and 
20%  involved  the  maxilla  or  zygoma  (i.e.,  sinuses). 

The  remainder  involved  the  external  ear  or 
tympanic  membranes.47 

Matthews,48  during  a  12-month  tour  of  duty  in  a 
field  hospital  in  Vietnam  analyzing  92  patients 
with  penetrating  wounds  to  the  brain  discovered 

that  these  wounds  not  infrequently  involve  the  paranasal  sinuses  and  orbit.  There  were  1 6  paranasal  sinus 
injuries  among  Matthews’s  92  cases  of  cranio-cerebral  trauma. 


Table  12.  Percent  distribution  for  head  versus  face 
and  neck  wounds  (KIA,  DOW,  and  living  wounded) 
for  U.S.  Army  in  WWII,  Korea,  and  Vietnam46 


War 

WWII 

Head 

Face  &  Neck 

Killed 

40 

9 

DOW 

20 

6 

Living  Wounded 

8 

8 

Korea 

Killed 

38 

10 

DOW 

25 

7 

Living  Wounded 

7 

11 

Vietnam 

Killed 

34-46 

8 

DOW 

46 

46 

Wounded 

17 

17 

44  Carey,  M.E.,  “An  Analysis  of  U.S.  Army  Combat  Mortality  and  Morbidity  Data,”  Journal  of  Trauma.  28  (1 , 
Suppl.),  SI  83-1 89, 1988,  Table  III. 

45  Rich,  N.M.,  Evaluation  of  missile  wounds  at  the  2nd  surgival  hospital  in  Vietnam,  Plastic  and  Maxillofacial 
Truama  Sympositum,  Vol.  1.,  St  Louis,  The  C.V.  Mosby  Co.,  1969,  p.  9. 

46  Evaluation  of  Wound  Data  and  Munitions  Effectiveness  in  Vietnam,  Volume  I”  Final  Report  for  the  Joint 
Technical  Coordinating  Group  for  Munitions  Effectiveness,  December  1970,  Table  I. 

47  Bellamy,  R.,  Personal  Communication,  16  June  1997. 

48  Matthews,  W.E.,  “The  Early  Treatment  of  Craniocerebral  Missile  Injuries:  Experience  with  92  Cases,”  Journal 
of  Trauma,  12(11):  939-954,  1972. 
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2.3.3  Regional  Conflicts 

Gofrit49  analyzes  405  penetrating  war  injuries  (290  shrapnel  and  115  bullet)  in  164  soldiers  killed  in  the 
Lebanon-Israeli  War.  The  body  part  with  the  greatest  density  of  penetrating  injuries  was  the  face,  with  22.2 
percent  of  all  penetrating  wounds.  In  particular,  the  mid-face,  from  the  level  of  the  lips  to  the  level  of  the 
zygomatic  bones,  was  especially  vulnerable,  sustaining  42  wounds,  which  was  10%  of  all  penetrating  wounds. 
Thirty-two  penetrating  wounds  were  found  in  the  upper  face,  above  the  zygomatic  bones,  and  19  in  the  lower 
face  below  the  level  of  the  mouth.  Table  13  shows  the  distribution  of  head  and  neck  wounds  from  Goffit's  study 
as  a  function  of  causative  agent. 


Table  13.  Distribution  of  head  and  neck  wounds  among  164  fatalities  in  Lebanon-Israeli  war 


Region 

Bullet 

Total  Hits 

Presented  Area 

Face 

EEXSEESIHHi 

3% 

Head 

19  (6.6%) 

17(14.8%) 

36  (8.9%) 

3% 

Neck 

10  (3.4) 

3% 

Total 

290 

115 

405 

3% 

Rautio  reported  on  200  Afghan  war  wounded  in  International  Red  Cross  Hospital  in  Quetta.  He  found  25  (12%) 
head  injuries,  of  which  14  had  penetrating  brain  trauma.  Of  nine  facial  injuries,  eight  were  relatively 
insignificant  and  one  suffered  a  gunshot  wound,  destroying  the  nose  and  upper  lip  and  fracturing  the  mandible 
and  hard  palate.50 

2.3.4  U.S.  Army  Aviators 

Three  studies,  two  from  the  Vietnam  era  and  one  from  WWII,  are  reviewed  in  this  section.  The  first  Vietnam- 
era  study  encompasses  2,946  wounds  occurring  between  1962  through  1967.51  The  face  represented  208  (7%) 
wounds,  and  with  the  exception  of  "unspecified  leg  wounds"  was  the  most  frequently  wounded  region  of  the 
body  during  this  interval.  Among  these  208  facial  wounds  were  30  eye  wounds.52  Tables  14  and  15 
summarize  these  results.  As  is  evident  from  these  tables,  debris  caused  most  of  the  wounds  but  was  a  negligible 
cause  of  fatal  and  serious  wounds.  Half  of  debris  resulted  from  Plexiglas  fragmentation.  Of 2,946  total  wounds 


Table  14. 1962-1967  -  Percentage  distribution  of  head  and  neck  wounds  by  cause  versus  location,  fatal 
and  serious  wounds53 


Anatomical  Region 

Bullet 

Aircraft  Debris 

Other  Causes 

Helmet  area 

34.0 

2.1 

0.0 

36.1 

22.7 

3.1 

0.0 

25.8 

15.5 

7.2 

2.1 

24.8 

Neck 

11.3 

1.0 

1.0 

13  3 

Total  Head  and  Neck 

83.5 

13.4 

3.1 

100.0 

49  Gofrit,  O.N.,  Kovalski,  N.,  etal.,  “Accurate  anatomical  location  of  war  injuries;  analysis  of  the  Lebanon  war  fatal 
casualties  and  the  proposition  of  new  principles  for  the  desing  of  military  personal  armour  system,”  Injury.  27(8): 
577-81,1996. 

50  Rautio,  J.  and  Paavolainen,  P.,  “Afghan  War  Wounded:  Experience  with  200  Cases,”  Journal  of  Trauma. 

28(4):  523-25,  1988. 

51  Bernier,  R.G.,  and  Smith  H.C.,  “U.S.  Army  Casualties  Aboard  Aircraft  in  the  Republic  of  Vietnam  (1967-1967), 
"Ballistic  Research  Laboratories  Memorandum  Report  No.  2030,  March  1970. 

52  Ibid.,  Table  D-22. 

53  Bernier,  Op  Cit.,  Table  4.29,  p.  68. 


73 


Appendix  D 


DAMD17-95-2-5023 
Final  Report 

Appendix  D  -  Trauma  Study 


Table  15. 1962-1967  -  Percentage  distribution  of  head  and  neck  wounds  by  cause  versus  location,  fatal 
and  serious  wounds55 


Anatomical  Region 

Bullet 

Aircraft  Debris 

Other  Causes 

Helmet  area 

10.0 

2.2 

3.1 

15.3 

6.4 

2.2 

1.4 

10.0 

6.0 

16.7 

34.1 

56.8 

Neck 

4.7 

3.3 

9.9 

17.9 

Total  Head  and  Neck 

27.1 

24.4 

48.5 

100.0 

between  1962  and  1967, 253  (8.58%)  were  due  to  Plexiglas  fragments.  Forty-six  of  these  wounds  (1.5%) 
affected  the  face  only.54 

The  second  Vietnam-era  study  encompasses  5,186  wounds  occurring  between  1968  through  1972.56  Of  these 
wounds,  446  (8.6%)  affected  the  face,  not  including  the  eyes,  and  399  (90%)  were  not  serious. 

In  this  study,  the  head  and  neck  was  initially  divided  into  forehead,  face,  skull,  ear  (mastoid),  neck,  and 
"unspecified."  However,  the  forehead,  skull,  and  ear  were  subsequently  grouped  into  the  "helmet  area"  for 
reporting  purposes.  In  this  casualty  population,  the  face  accounted  for  over  half  of  the  head  and  neck  wounds, 
but  only  25%  of  the  fatal  wounds  and  15%  of  the  serious  wounds.57  As  in  the  previous  study,  facial  wounds,  not 
including  eye  injuries,  and  with  the  exception  of  "unspecified  leg  wounds,"  were  the  most  frequently  sustained 
wound  of  the  body  during  this  interval. 

Finally,  Palmer,  in  a  study  of  U.S.  aviators  during  WWD,  documented  the 
striking  locations  of  85  penetrating  facial  wounds  from  Plexiglas 
fragments  occurring  in  75  WIA  casualties.58  See  Figure  14.59 


Figure  14.  Location  of  85 
wounds  due  to  Plexiglas 
fragments  in  75  WIA 
casualties 


54  Ibid.,  Table  4.20,  p.  61. 

55  Bernier,  Op  Cit.,  Table  4.29,  p.  68. 


56  Malick,  D.,  “U.S.  Army  Casualties  Aboard  Aircraft  in  the  Republic  of  Vietnam  (1968  through  1970),"  Ballistic 
Research  Laboratory  Contract  Report  257,  August  1975. 

57  Ibid.,  p.  97. 

58  Palmer,  A.,  "Survey  of  Battle  Casualties,  Eighth  Air  Force,  June,  July,  and  August  1941,"  Chapter  IX  in: 
Coates,  J.B.,  and  Beyer,  J.C.,  Wound  Ballistics,  Medical  Department,  United  States  Army,  Government  Printing 
Office,  Washington,  D.C.,  December  1962. 

59  Ibid. 
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3  TASK 2 -TISSUE  RESPONSE  AND  DAMAGE 

This  section  is  divided  into  two  subsections.  The  first  discusses  gross  characteristics  of  tissue  damage 
associated  with  various  categories  of  small  arms  and  the  second  provides  several  supporting  case  histories. 

3.1  Gross  Characteristics  of  Tissue  Damage  Associated  with  Various  Categories  of  Small  Arms 

As  previously  mentioned,  there  are  some  gross  characteristics  of  penetrating  wounds  that  can  be  associated  with 
various  classes  of  weapons.  At  the  less  severe  end  of  the  spectrum  are  low-velocity  missile  threats  that  include 
handguns,  airguns,  and  low-velocity  fragments  from  spall  and  strikes  by  shrapnel  or  fragments  from  a  long 
distance  following  detonation  of  a  high-energy  munition.  At  the  severe  end  of  the  spectrum  are  high-velocity 
missile  threats  that  include  military  rifles  and  strikes  by  shrapnel  or  fragments  in  close  proximity  to  an 
exploding  mortar  or  artillery  shell.  Shotguns,  which  will  be  discussed  separately,  encompass  the  entire  spectrum 
of  injury,  depending  on  distance,  choke,  and  projectiles  fired. 

3.1.1  Low-velocity  proj  ectiles 

Low-velocity  projectiles  cause  laceration  and  crushing  of  tissues.  The  surrounding  structures  are  not  usually 
affected  and  injuries  are  generally  not  serious  unless  a  vital  organ  or  vessel  is  directly  struck.  Both  the  entrance 
hole  and  residual  channel  are  smaller  than  the  diameter  of  the  missile  because  of  elastic  recoil  of  the  soft 
tissues.  An  exit  wound,  if  present,  is  often  ragged  and  slit-shaped  due  to  yaw  of  the  projectile  in  the  tissues,  or 
fragmentation  if  bone  or  teeth  have  been  encountered.  Low-velocity  bullets  are  usually  made  of  lead  alloy  and 
deform  and  partially  disintegrate  in  tissue.  Sometimes  these  bullets  are  surrounded  by  a  copper  or  cupronickel 
jacket  which  may  become  dislodged  or  break  up  in  the  body. 

In  a  retrospective  analysis  by  Cohen  and  Shakenovsky  of  40  cases  of  low-velocity  handgun  injuries  of  the 
maxillofacial  region,  due  to  assault  followed  by  accidental  shootings  and  failed  suicide  attempts;  injuries 
ranged  from  mild  to  soft  tissue  damage  without  fractures  to  severe,  comminuted  fractures  of  the  facial  bones.  In 
some  cases  soft  tissue  damage  was  severe.  Rarely  were  injuries  life  threatening  or  even  mutilating  and  entrance 
wounds  were  characteristically  small  and  well  circumscribed.  In  60%  of  the  cases  the  bullet  did  not  exit  and 
was  retained  within  the  tissues. 60 

The  characteristic  feature  of  low-velocity  wounds  is  that  the  injury  is  confined  to  the  permanent  wound  tract 
and  to  tissues  and  organs  directly  penetrated  by  the  missile.  Airguns,  for  example  usually  result  in  little 
morbidity  unless  a  delicate  structure  such  as  the  eye  is  perforated.  Due  to  the  elasticity  of  the  soft  tissues,  the 
narrow  wound  channel  closes  spontaneously  with  little  damage  to  adjacent  structures.  When  the  dense  bone  of 
the  mandible  is  encountered,  the  pellet  is  flattened  but  does  not  usually  have  sufficient  energy  to  fracture  the 
bone  (see  Figure  2).  However,  an  airgun  pellet  may  penetrate  the  thin  bone  of  the  maxilla  and  enter  the 
maxillary  antrum  or  nasal  cavity. 

On  impact  with  the  mandible  by  a  low-velocity  projectile,  the  bone  is  often  comminuted,  but  the  projectile 
usually  has  insufficient  energy  to  expel  the  fragments  which  are  held  in  place  by  the  periosteum  and  soft 
tissues.  The  thin  bones  of  the  maxilla  are  more  susceptible  to  injury  and  may  be  severely  comminuted  and 
displaced  around  the  missile  track.  Gunshot  wounds  of  the  maxilla  produce  fractures  that  rarely  follow  the 
classic  Le  Fort  pattern,  but  alveolar  fracturing  is  common  with  associated  oroantral  and  oronasal 
communication.  Low-velocity  gunshot  wounds,  however,  tend  to  penetrate  and  divide  maxillary  bone  tissue  and 


60  Cohen,  M.A.,  Shakenovsky,  B.N.,  et  al.,  “Low  Velocity  Hand-Gun  Injuries  of  the  Maxillofacial  Region,”  Journal 
of  Maxillofacial  Surgery,  14:26-33, 1986. 
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fracture  the  cortical  surface  of  the  mandible.  Teeth,  dentures  and  the  projectile  itself  may  also  fragment  and  be 
widely  displaced  in  surrounding  tissue. 

In  many  cases,  the  bullet  is  deflected  off  bony  structures  at  a  tangent  and  follows  an  irregular  pathway. 
Frequently,  low-velocity  missiles  from  handguns  break  up  and  leave  small  metal  particles  in  the  projectile’s 
path.  The  pathway  may  often  be  seen  radiologically  by  the  presence  of  lead  fragments  along  the  bullet  track. 
Low-velocity  missiles  may  also  tend  to  be  deflected  by  such  structures  as  vessels,  nerves,  and  facial  planes.  The 
path  of  these  slowly  moving  projectiles  will  therefore  be  much  more  erratic,  sometimes  having  no  relation  to 
entrance  and  exit  wounds.  Dr.  Robert  McCormack  (Rochester,  NY)  in  the  Comments  on  Discussion  section  of 
Gant's  study  on  low-velocity  gunshot  wounds  to  the  maxillofacial  complex61  makes  the  following  clinical 
observation:  “We  had  very  many  instances  of  extremely  bizarre  pathways."  Entrance  and  exit  wounds  do  not 
necessarily  go  in  a  straight  line  for  these  wounds  since  the  bony  structures  of  the  maxillofacial  complex  cause 
ricocheting  effects.  In  contrast,  high-velocity  missiles  cut  straight  through  soft  tissues  and  fracture  bones  with 
relative  ease. 

The  amount  of  tissue  injury  produced  by  these  low-velocity  missiles  is  considerably  less  than  that  from  high- 
velocity  weapons,  which  is  an  important  consideration  in  the  management  of  these  injuries.  The  entrance 
wound  usually  is  smaller  than  the  diameter  of  the  bullet  and  expected  to  heal  without  discernible  scarring.  The 
rich  blood  supply  of  the  face  allows  conservative  debridement  of  the  soft  tissue  injury.  The  infection  rate  is  low 
with  civilian  gunshot  wounds  in  contrast  to  high-velocity  bullet  injuries,  which  require  aggressive  debridement 
of  devitalized  tissue  and  delayed  wound  repair. 

In  contradistinction  to  high-velocity  missiles,  low-velocity  missiles  also  do  not  commonly  cause  significant 
vascular  trauma.  The  missile  will  commonly  push  nerves  out  of  its  pathway  as  they  are  only  loosely  fixed  in 
tissue  and  have  a  great  deal  of  elasticity. 

Unique  contours  and  variation  in  properties  of  the  bony  tissues  govern  the  interaction  of  penetrating  projectiles 
with  midfacial  structures.  Nahum  derived  tolerance  levels  of  the  facial  bones  from  a  series  of  impact 
experiments  on  cadavers.  Selected 
facial  bones  were  impacted  with 
varying  forces.  A  total  of  1 16  impact 
experiments  were  performed  with  the 
following  ranges  of  tolerance  levels 
for  facial  bones:  nasal  bones  —  20  to 
75  lb,  maxilla  -  150  to  300  lb, 
zygoma  -  200  to  650  lb,  and  the 
mandible  -  550  to  900  lb.  These 
results  are  represented  in  Figure  15.62 
Similar  results  are  shown  in  gravity 
units  (G's)  in  Figure  16  from 
Swearington.63 

Figure  15.  Ranges  of  tolerance  levels  for  facial  fractures  (lb) 


61  Gant,  T.D.,  and  Epstein,  LI.,  “Low-velocity  Gunshot  Wounds  to  the  Maxillofacial  Complex,"  Journal  of  Trauma. 
19(9):  674-77, 1979. 

62  Nahum,  A.M.,  "The  Biomechanics  of  Maxillofacial  Trauma,"  Clinics  in  Plastic  Surgery,  (2)1:  59-64,  January 
1975,  Figure  7,  p.  63. 

63  Swerington,  J.J.,  'Tolerance  of  the  Human  Face  to  Crash  Impact,"  Reprint  #65-20,  Federal  Aviation  Agency, 
Oklahoma  City,  OK,  1964. 
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3.1.2  High-velocity  proj  ectiles 

High-velocity  projectiles  cause  much  more  disruption  of  the  tissue  than  low- 
velocity  projectiles.  A  wound  from  a  high-powered  military  rifle,  for  example, 
is  characterized  by  an  avulsive  wound  with  massive  tissue  destruction  of  soft 
and  osseous  tissue.  The  entrance  wound  is  a  gaping,  devastated  area  of  soft 
tissue,  often  with  total  avulsion  of  hard  tissue.  Impact  of  a  high-velocity 
missile  on  bone  and  teeth  will  produce  numerous  secondary  missiles,  all  of 
which  have  the  potential  for  high-velocity  movement  within  tissues. 

The  effect  of  cavitation  on  the  living  body  is  to  pulp  the  soft  tissues,  disrupt 
small  blood  vessels,  and  produce  extensive  areas  of  necrosis.  Larger  blood 
vessels  and  nerves  are  elastic  and  may  be  pushed  aside,  but  the  intima  may  be 
damaged  even  though  there  is  no  external  sign  of  injury,  and  nerve  axis 
cylinders  may  disintegrate.  Thrombosis  and  stasis  in  vessels  further  increase 
the  extent  of  the  necrosis  and  leakage  of  plasma  causes  tense  edema.  However, 
the  soft  tissue  bulk  of  the  face  is  smaller  and  has  a  better  blood  supply  than 
other  parts  of  the  body  and  is  less  sensitive  to  those  destructive  effects. 

The  underlying  intricate  bony  architecture  of  the  maxillofacial  region  is  very  susceptible  to  damage  by  high- 
velocity  missiles,  whether  by  direct  contact  or  by  the  outward  expansion  of  the  temporary  cavity.  Fragments  of 
calcified  structures  may  be  scattered  widely  within  the  tissues,  or  be  expelled  from  the  body,  leaving  a  large 
ragged  exit  wound.  High-velocity  projectiles  are  rarely  retained  intact  in  the  facial  region  unless  they  have  been 
impeded  by  previous  contact  with  other  structures  or  by  ricochet.  Jacketed  bullets  usually  pass  through  the  body 
without  changing  shape,  but  the  jacket,  if  thin,  occasionally  fragments  into  small  pieces  even  on  impact  with 
soft  tissues  and  may  be  widely  scattered  to  the  full  extent  of  the  temporary  cavity. 

3.1.3  Shotgun 

Civilian  shotgun  injuries  are  also  characterized  by  severe  soft  and  hard  tissue  loss,  with  gross  mutilation  of  the 
affected  structures.  In  a  retrospective  review  by  Sherman  and  Parrish64  of  534  patients  with  gunshot  wounds 
admitted  to  John  Gaston  Hospital  (Memphis)  from  1948  through  1959  (patients  who  died  shortly  after 
admission  were  excluded),  152  patients  had  shotgun  wounds  and  382  patients  had  wounds  produced  by  a 
variety  of  other  firearms.  Sherman  and  Parrish  found  that  the  mortality  from  shotgun  wounds  was  more  than 
twice  that  of  other  gunshot  wounds.  In  this  study,  89  of  152  shotgun  injuries  were  to  the  head,  neck,  and 
extremities,  compared  to  163  of  the  382  gunshot  wounds. 

In  general,  Sherman  and  Parrish  found  that  at  ranges  exceeding  7  yards,  wounds  were  penetrating  with 
projectile  penetration  of  subcutaneous  tissue  or  deep  fascia.  At  3-7  yards,  wounds  were  perforating  with 
projectiles  perforating  deep  fascia  into  body  cavities.  At  ranges  under  3  yards,  the  wounds  manifested  extensive 
mutilating  tissue  destruction. 

Wounds  from  a  shotgun  fired  from  over  20  feet  are  similar  to  those  of  multiple  slow-velocity  weapons.  On  the 
other  hand,  shotgun  blasts  at  close  range  are  more  like  high-velocity  weapons.  As  the  range  decreases,  the 
pattern  of  pellets  striking  the  target  will  have  a  smaller  diameter.  At  very  close  range,  the  size  of  the  pellets 
makes  little  difference,  since  they  act  as  one  large  mass  traveling  at  a  high  velocity.  It  is  also  important  to 
realize  that  a  shotgun’s  wadding  acts  as  a  separate  missile  at  close  range  and  may  escape  detection  since  it  is 
radiotranslucent. 

64  Sherman,  R.T.,  and  Parrish,  R.A.,  “Management  of  Shotgun  Injuries:  A  Review  of  152  Cases,"  Journal  of 
Trauma "  3:  76-86, 1963. 


Figure  16.  Threshold 
accelerations  required  to 
fracture  selected  facial 
bones,  in  gravity  units  (G's) 
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In  self-inflicted  wounds  from  shotguns  and  rifles  that  are  long  barreled,  the  gun  is  often  placed  under  the  chin. 
This  requires  the  head  to  be  tilted  backwards  a  fair  distance  for  the  patient  to  be  able  to  reach  the  firing 
mechanism  and  discharge  the  gun.  This  commonly  results  in  massive  loss  of  the  mandible,  lower  lip,  floor  of 
the  mouth,  and  tongue,  and  may  result  in  massive  loss  of  the  palate,  maxilla,  nose,  nasal  passages,  and  one  or 
both  eyes.  Unless  placed  in  the  mouth,  however,  intracranial  involvement  is  uncommon. 

3.2  Selected  Case  Histories 

The  following  case  history  is  reproduced  from  Mektubjian65  concerning  a  wound  from  a  low-velocity  handgun. 
A  26-year-old  female  patient  was  struck  in  the  face  by  a  bullet  from  a  small-bore  pistol  at  a  distance  of  one 
meter.  The  entrance  wound  was  situated  behind  the  right  nasolabial  sulcus,  with  the  exit  wound  in  the 
vestibulum  oris  in  the  area  of  upper  first  molar  tooth.  The  crowns  of  the  upper  and  lower  right  molars  were 
fractured  off,  comminuted  and  embedded  through  multiple  puncture  wounds  in  the  root  of  the  tongue  and  the 
floor  of  the  oral  cavity.  Tissue  lacerations  were  present  along  the  right  mandibulo-lingual  sulcus  and 
pterygomandibular  fold.  The  continuity  of  the  mandible  was  not  interrupted.  Two  large  fragments  from  the 
inner  lingual  cortex  of  the  angle  area  of  the  mandible,  and  portions  of  the  alveolar  process  were  embedded  in 
the  neighboring  soft  tissues. 

The  lateral  and  P-A  X-rays  of  the  facial  skeleton  revealed  numerous  metallic  foreign  bodies  varying  in  shape 
and  size,  grouped  in  the  right  half  of  the  mandible,  and  one  -  the  largest  -  projected  along  the  left  lower 
mandibular  border.  The  fractured  crowns  of  the  right  upper  molars  and  the  scattered  shadows  of  fragments  of 
teeth  and  displaced  roots  were  visible. 

In  the  course  of  a  further  X-ray  study,  a  defect  in  the  body  of  the  third  cervical  vertebra  was  discovered.  The 
projectile  had  deflected  twice,  once  on  touching  the  mandibular  bone,  and  a  second  time,  by  the  supero-anterior 
edge  of  the  third  cervical  vertebra. 

On  the  tenth  day,  a  greatly  deformed  lead  bullet  was  removed.  It  was  lodged  beneath  the  sternocleidomastoid 
muscle,  laterally  and  in  close  proximity  to  the  neurovascular  bundle  of  the  neck,  which  showed  no  evidence  of 
traumatic  change. 

Facial  wounds  are  fatal  when  hemorrhage  causes  aspiration,  or  displacement  of  the  severed  structures  causes 
airway  obstruction.  A  representative  case  is  that  of  a  man  wounded  by  a  bullet  that  entered  the  left  jaw, 
transected  the  tongue,  and  fractured  the  maxilla  and  mandible.  The  exit  wound  was  on  the  right  side  of  the  face. 
The  wound  tract  measured  285  mm.  Aspiration  of  blood  caused  death.66 

Bostwick67  in  his  retrospective  study  of  547  patients  treated  at  Grady  Hospital  for  penetrating  wounds  to  the 
face  and  neck  (refer  to  Section  2.2.2),  discovered  only  one  death  from  facial  injury  due  to  upper  airway 
obstruction  from  rapidly  expanding  retro  pharyngeal  hematoma  not  managed  by  tracheostomy  or  exploration. 
Active  bleeding  from  a  vascular  injuiy  to  the  face  can  be  significant  because  of  the  diffusely  vascular  nature  of 
the  facial  structures  and  the  presence  of  major  blood  vessels  within  the  upper  neck.  Avulsed  or  fragmented  teeth 
and/or  dentures  can  also  result  in  aspiration  of  teeth  with  subsequent  laryngospasm  or  bronchial  obstruction. 


65  Mektubjian,  S.R.,  “Low  Velocity  Gunshot  Maxillofacial  Injury  Combined  with  a  'Blind'  Wound  of  the  Neck,” 
Journal  of  Maxilliofacial  Surgery,  9:  85-55, 1981. 

66  “Evaluation  of  Wound  Data  and  Munitions  Effectiveness  in  Vietnam,  Volume  I”  Final  Report  for  the  Joint 
Technical  Coordinating  Group  for  Munitions  Effectiveness,  December  1970,  p.  2-14. 

67  Bostwick  J.,  Schneider,  W.J.,  et  al.,  “Penetrating  Injuries  of  the  Face  and  Neck,”  Southern  Medical  Journal, 
76(69):  550-553,  May  1976. 
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Upper  facial  bony  structures  may  afford  considerable  protection  from  intracranial  involvement  and  low-velocity 
projectiles.  Gussack  reports  of  a  12-year  boy  who  was  shot  in  the  mid-forehead  with  a  pellet  rifle;  however, 
there  was  no  intracranial  penetration  and  the  pellet  was  lodged  in  frontal  sinus.68  The  WDMET69  reports  of  a 
casualty  walking  on  a  sweep  maneuver  through  the  jungle  when  a  fragment  struck  him  from  a  Claymore  mine. 
His  helmet  was  not  hit.  He  sustained  a  comminuted  fracture  of  the  frontal  sinus,  but  there  was  no  dural 
penetration. 

Dillon70  also  reports  of  a  case  involving  a  fragmentation  wound  to  the  facio-orbito-frontal  region.  The  projectile 
entered  the  inner  canthus  of  the  right  eye.  The  missile  passed  through  the  right  orbit  and  frontal  sinus.  It  then 
penetrated  the  floor  of  the  right  anterior  fossa,  lodging  in  the  right  frontal  lobe.  There  was  rhinorrhea  as  well  as 
cerebrospinal  fluid  leakage  from  the  entiy  site.  Loss  of  cerebrospinal  fluid  from  the  nose  indicates  that  there  is 
an  open  road  for  bacteria  or  air  to  the  meninges  and  brain.  Cerebrospinal  rhinorrhea  is  seen  most  often  in 
wounds  about  the  frontal  sinuses;  however,  its  immediate  recognition  is  often  obscured  by  the  profuse  bleeding 
accompanying  these  wounds.  In  this  case,  a  metallic  fragment  was  removed  from  the  frontal  sinus. 

4  TASK  4 -TRAUMA  MODEL  DEVELOPMENT 

Given  the  extreme  variability  of  facial  wounds  and  the  associated  projectile  interactions,  an  important  feature  of 
a  surgical  training  simulator  for  these  wounds  would  be  to  recreate  some  of  this  variability  as  well  as  the  non- 
visible  mechanical  damage  that  occurs.  Unfortunately,  this  requires  either  a  database  of  wounds  that  have  been 
cross-sectioned  and  digitized,  (which  does  not  exist)  or  a  set  of  models  that  describes  or  infers  the  relevant 
phenomenology  at  an  acceptable  level  of  detail  and  fidelity. 

Development  of  maxillofacial  penetrating  wound  models  affecting  the  paranasal  sinuses  should  proceed  in  three 
stages.  The  first  stage  would  address  low-velocity  projectiles  with  striking  locations  in  the  maxillofacial  region 
in  close  proximity  to  the  paranasal  sinuses  where  the  projectile  is  retained  in  soft  tissue.  During  this  initial 
stage,  bony  structures  would  be  modeled  as  rigid  and  the  projectile  would  be  non-deforming.  The  focus  on  this 
initial  phase  would  be  to  model  the  projectile  trajectory  in  soft  facial  tissue,  bleeding  and  swelling,  and  create 
an  interface  with  the  anatomical  model  used  by  the  simulator.  This  interface  would  map  a  database  of  relevant 
material  properties  and  provide  a  Boolean  algebra  to  remove  and/or  modify  regions  of  polygons  in  the 
simulator.  One  idea  would  be  to  create  regions  of  polygons  that  overlay  the  initial  anatomy  of  the  simulator  but 
are  not  displayed  until  a  flag  is  registered. 

During  the  second  phase,  bone  interaction  would  be  introduced.  Initially,  dynamic  fracture  thresholds  of 
relevant  bony  features  would  be  approximated  in  terms  of  energy  density  per  unit  time  thresholds.  This  would 
necessarily  be  determined  experimentally  and  would  require  embalmed  human  facial  bones  (or  skulls)  to  be 
supplied  as  GFE.  Existing  data  on  dynamic  bone  fracture  of  facial  structures  has  been  developed  almost 
exclusively  to  support  analysis  of  automobile  crashes.  In  general  this  data  is  not  useful  for  ballistic  impact  since 
the  pulse  width  of  the  forcing  function  in  both  cases  differs  by  at  least  two  orders  of  magnitude.  In  fact,  the 
governing  parameters  for  these  two  scenarios  are  probably  entirely  different  in  that  the  ballistic  damage  is 
impulse  driven,  whereas  the  crash  scenario  is  driven  by  stress/strain  excursions  in  the  bony  structures. 

Ballistic  experiments  would  have  to  be  implemented  where  energy  density  would  be  measured  and  related  to 
fracture  mode.  The  scale,  distribution,  and  ejection  velocity  of  bone  fragments  would  be  measured  in  a  skeletal 


68  Gussack,  G.S.,  and  Jurkovich,  G.J.,  “Penetrating  Facial  Trauma:  A  Management  Plan.  Southern  Medical 
Journal ,”  81(3):  297-302, 1988,  Case  3. 
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muscle  tissue  simulant  as  well  as  projectile  penetration  thresholds  and  residual  exit  velocities.  Fragment 
geometry,  distribution,  and  initial  velocities  would  then  be  modeled  stochastically  based  on  experimental 
results.  This  will  result  in  relevant  features  of  the  wound  presentation  being  continually  different  to  the  end 
user.  Ideally,  the  skulls  would  be  from  soldier-aged  populations,  x-rayed  to  determine  thickness  of  cortical 
layers.  A  separate  series  of  ballistic  experiments  would  be  implemented  to  refine  and  verify  existing  bone 
deflection  models  developed  at  Mission  Research  Corporation  (MRC).  At  this  stage  of  the  modeling,  the 
striking  location  of  the  projectile  would  still  be  in  the  vicinity  of  the  maxillaiy  complex.  The  emphasis  of  the 
modeling  effort  at  this  stage  would  be  to  model  the  three-dimensional  non-rigid  bone  interaction  and  rigid  body 
kinematics  of  the  projectile  that  is  still  modeled  as  non-deforming. 

The  third  stage  of  the  modeling  effort  would  relax  the  assumption  of  a  non-deforming  projectile  and  permit 
projectile  striking  locations  anywhere  on  the  face  that  could  affect  the  paranasal  sinuses.  Equations  of  state 
(EOSs)  would  be  developed  for  relevant  bony  structures  on  the  face.  The  EOSs  would  be  used  in  a  hydrocode 
to  determine  relevant  wound  features  and  parametric  sensitivities.  Results  form  these  hydrocode  models  would 
be  formatted  as  a  library  of  base  models  with  associated  "look  up"  tables  to  perturb  the  base  models.  Since  the 
overall  thrust  of  this  effort  is  to  interface  with  an  endoscopic  simulator,  it  is  probably  not  relevant  to  model 
avulsion  of  tissue  or  interaction  of  high-velocity  fragments  or  military  rifle  bullets  (particularly  when  it  is 
considered  that  overwhelmingly,  penetrating  wounds  from  military  rifles  occur  at  close  range,  under  20  yards). 

The  first  phase  of  the  proposed  effort  is  approximately  5  man-years  over  18  months.  The  second  phase  is  on  the 
order  of  9  man-years  over  30  months,  and  the  final  phase  is  approximately  5  man-years  over  18  months.  Many 
aspects  of  the  proposed  effort  necessarily  advance  the  state  of  the  art  relative  to  the  biomechanics  of  bone 
fracture.  A  literature  review  for  another  project  revealed  that  there  are  only  12  articles  published  in  the  word 
literature  relative  to  ballistic  trauma  to  bones.  The  most  recent  article  was  published  in  the  mid-1980's  in  the 
People's  Republic  of  China.  In  all  cases,  the  reviewed  articles  dealt  with  fracture  of  long  bones  by  ballistic 
events.  Therefore,  in  many  respects  the  proposed  work  will  be  unique. 
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ABSTRACT 

The  Madigan  Endoscopic  Sinus  Surgery  (ESS)  Simulator  is  a  state-of-the-art 
procedural  simulator  developed  by  a  multi-institution  team  led  by  researchers 
at  Lockheed  Martin.  It  includes  force-feedback  instrument  and  virtual 
endoscope  interfaces  developed  by  Immersion  Corporation,  and  3-D  models  of 
the  paranasal  sinus  anatomy  derived  from  the  Visible  Human  dataset  by 
researchers  at  the  Ohio  Supercomputer  Center,  supplemented  by  a  variety  of 
instructional  aids,  such  as  hoops  for  navigation  paths  and  targets  for  injection 
sites.  We  describe  here  the  results  of  our  formal  evaluation  of  the  simulator,  led 
by  researchers  at  the  Human  Interface  Technology  (HIT)  Laboratory  at  The 
University  of  Washington.  Formative  evaluation  efforts  throughout  the 
development  phase  of  the  project  informed  the  system  design  and  user 
interface  requirements  for  the  simulator.  Summative  evaluation  of  Version  1.2 
of  the  system  was  conducted  during  the  second  phase  of  the  project,  with  an 
emphasis  on  its  validity  as  an  ESS  simulator.  Run-time  and  survey  data  were 
collected  on  three  groups  of  subjects  who  ran  through  a  common  protocol 
which  progressed  through  three  basic  surgical  tasks:  navigation,  injection,  and 
dissection.  A  group  of  non-physician  subjects  performed  the  tasks  on  an 
abstract  virtual  model  with  the  instructional  aids.  A  group  of  non¬ 
otolaryngologist  MD  subjects  progressed  through  this  abstract  model  to  an 
intermediate  model  with  the  instructional  aids  embedded  in  the  paranasal 
anatomy.  A  group  of  otolaryngologists  ranging  from  second-year  residents  to 
experienced  senior  staff  ran  through  the  abstract  and  intermediate  models, 
followed  by  an  advanced  model  of  the  anatomy  with  no  instructional  aids.  The 
procedural  validity  of  the  simulator  is  supported  by  the  strong  correlation 
between  performance  on  the  ESS  simulator  and  degree  of  prior  ESS  experience, 
and  by  the  subjective  evaluations  by  the  experienced  ESS  surgeons.  A  novel 
path  visualization  technique  is  also  introduced,  with  a  set  of  quantitative 
measures  of  endoscope  and  instrument  path  performance.  These  measures 
provide  initial  evidence  for  the  validity  of  the  system-generated  scoring 
algorithms  based  on  time,  completeness,  and  accuracy. 
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1  INTRODUCTION 

The  use  of  computer  simulation  as  a  training  tool  for  surgical  procedures  has  been  motivated  by  the 
success  of  computer  simulation  flight  training  as  well  as  the  high  cost  of  medical  education.  In 
recent  years,  advances  in  interactive  graphics  and  virtual  reality  technology  have  greatly  enhanced 
our  arsenal  of  instructional  tools  (Caird,  1996),  moving  these  systems  toward  more  general 
commercial  graphics  platforms. 

A  recent  survey  of  prior  research  in  surgical  simulation  suggests  that  each  surgical  procedure  has  a 
unique  set  of  simulation  requirements  (Edmonds  et  al.,  1997).  Research  thus  far  includes  abdominal 
laparoscopy  (Cover  et  al.,  1993;  Hon,  1994;  Merril,  1994;  McGovern  et  al.,  1994);  limb  surgery  (Chen 
et  al.,  1992;  Pieper  et  al.,  1991);  eye  surgery  (Peifer  et  al.,  1994;  Sagar  et  al.,  1994);  plastic  surgery 
(Pieper,  1989);  gastrointestinal  endoscopy  (Baillie  et  al.,  1992;  Bard,  1990;  Gillies  et  al.,  1992;  Poon  et 
al.,  1988);  anesthesiology  (Good  et  al.,  1993);  epidural  anesthesiology  (Bostrom  et  al.,  1993;  Stredney 
et  al.,  1996);  and  interventional  radiology  procedures  (Dawson  et  al.,  1996).  Much  of  this  recent  work 
in  medical  simulation  has  received  impetus  from  the  leadership  of  both  DARPA  (Satava,  1996)  and 
the  Army  Medical  Command. 

Wickham  (1994)  summarizes  the  need  for  these  novel  and  extensive  training  techniques  for 
endoscopic  surgery  skills: 

Evaluation  of  new  operative  competence  is  urgently  needed  because  of  the  rapidity  of 
changes  in  interventional  treatment.  Training  programmes  must  be  established  so 
that  interventionists'  training  is  similar  to  that  of  airline  pilots.  A  surgeon  or 
radiologist  should  not  be  allowed  to  treat  patients  with  sophisticated  and  potentially 
dangerous  instruments  without  the  experience  of  simulated  operations  and  closely 
supervised  procedural  training.  Fully  equipped  training  centers  should  be  established 
with  simulator  laboratories  where  interventionists  can  develop  the  different  hand-eye 
coordination  required  for  the  transition  from  open  to  endoscopic  techniques  .  .  .  The 
need  is  urgent:  the  traditional  methods  of  "see  one,  watch  a  video,  do  one"  are 
completely  inadequate  preparation  for  minimally  invasive  techniques  ...  A  theoretical 
evaluation  of  competence  by  written  or  oral  examination  is  totally  insufficient  to 
determine  whether  a  clinician  has  gained  the  manual  ability  to  carry  out  complex  open 
or  endoscopic  surgery. 

The  motivation  for  medical  training  simulators  is  clear.  It  is  important,  however,  to  ask:  Do  they 
work?  Is  the  simulator  effective  as  a  training  device?  What  skills  do  students  leam  most  effectively? 
Do  the  skills  learned  in  the  simulated  training  transfer  to  the  operating  room?  How  fast  do  students 
leam?  How  does  an  hour  of  simulator  time  compare  to  an  hour  of  traditional  training  methods?  Are 
these  simulators  currently  cost  effective?  If  not,  when  will  they  be  cost  effective  for  use  in  medical 
schools?  And  how  can  simulators  best  be  integrated  with  medical  school  curriculum? 

To  answer  these  questions,  formal  evaluation  of  surgical  simulation  systems  is  essential.  Without 
answers,  one  remains  uncertain  as  to  whether  students  are  receiving  surgical  training  or  merely 
learning  to  be  good  simulator  users. 

The  evaluation  study  described  here  has  been  guided  by  the  work  of  many  others  in  the  field.  As 
Hoffman  et  al.  (1996)  suggest,  we  have  included  the  end  users  during  the  formative  stages  of  the 
simulator  design,  a  vital  step  in  establishing  educational  goals  and  curriculum  design.  In  developing 
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our  evaluation  criteria  we  have  taken  into  account  both  objective  and  subjective  considerations 
(Robb,  1997),  relied  heavily  on  the  basic  surgical  proficiency  measures  of  time  and  accuracy  (Johnson 
et  al.,  1996),  and  incorporated  much  of  our  evaluation  protocol  into  the  simulator  itself  (Hon,  1996; 
Kaufmann,  1997). 

1.1  Simulator  Overview 

The  Madigan  Endoscopic  Sinus  Surgery  (ESS)  Simulator  uses  virtual  reality  technology,  a  force 
feedback  (haptic)  display,  and  3D  computer-based  anatomy  models  as  a  tool  to  teach  a  variety  of 
skills  needed  to  perform  such  surgery. 

The  heart  of  the  system  is  a  3D  model  of  the  human  nasal  sinus  anatomy  derived  from  the  National 
Library  of  Medicine's  Visible  Human  Database.  Researchers  at  the  Ohio  Supercomputer  Center 
(OSC)  created  segmented  surface  models  of  the  sinus  anatomy  from  photographic  cryosections  of 
the  male  dataset.  The  Lockheed  Martin  team  then  added  stochastically  generated  surface  textures  to 
complete  the  anatomical  model. 

This  model  can  be  rendered  in  real  time  (15-30  frames  per  second,  depending  primarily  on 
anatomical  position)  on  a  Silicon  Graphics  (SGI)  Onyx  System  and  viewed  at  NTSC  resolution  on  a 
standard  video  monitor,  thus  simulating  the  view  that  a  surgeon  would  see  of  a  video  endoscopic 
display. 

To  interact  with  the  model  and  perform  the  simulated  surgery,  students  in  training  use  a  pair  of  6- 
degree-of-freedom  input  devices  developed  by  Immersion  Corporation.  One  such  device  represents 
the  video  endoscope  and  the  other  represents  the  surgical  instrument,  such  as  an  injection  needle  or 
a  forceps  (Rosenberg  and  Stredney,  1996).  An  external  view  of  an  experienced  ENT  surgeon 
operating  the  system  can  be  seen  in  Figure  1.  Note  that  the  position  of  the  surgeon  relative  to  the 
endoscopic  monitor  and  simulated  patient  emulates  a  typical  clinical  configuration. 


Figure  1.  Operative  Configuration  of  the  Madigan  Endoscopic  Sinus  Surgery  Simulator 


87 


Appendix  E 


DAMD17-95-2-5023 
Final  Report 

Appendix  E  -  Formal  Evaluation 

As  the  student  manipulates  the  input  devices,  the  simulator  tracks  the  position  and  orientation  of  the 
devices,  updates  the  positions  and  orientations  of  the  virtual  endoscope  and  virtual  instrument, 
manipulates  the  virtual  anatomical  model  accordingly,  and  displays  the  resulting  virtual  endoscopic 
view  on  the  monitor.  In  addition,  the  system  tracks  the  opening  of  the  forceps  handle  of  the 
instrument  input  device.  All  together,  the  system  measures  13  degrees  of  freedom  of  the  student’s 
input. 

The  physical  input  devices  are  designed  to  resemble  the  feel  of  an  endoscope  and  forceps  and  are 
assembled  with  a  latex  replica  of  a  human  head.  The  endoscope  input  device  resides  outside  the 
head,  while  the  instrument  input  device  is  inserted  into  the  nostril  of  the  latex  head  and  attached  to 
the  position  tracker  inside  the  head.  In  addition  to  displaying  the  simulated  endoscopic  view,  the 
system  computes  the  forces  that  the  sinus  tissue  would  apply  to  the  tip  of  the  instrument  during 
surgery  and  applies  the  computed  force  to  the  instrument  input  device,  via  mechanical  coupling 
within  the  mannequin's  head. 

The  proctor's  console  provides  an  interface  to  the  run-time  system  parameters,  along  with  optional 
radiographic  views  of  the  current  anatomy  and  optional  performance  feedback  summaries  for  the 
trainees. 


Training  Aids 

Optional  3D  graphical  overlays  are  superimposed  on  the  endoscopic  view  to  provide  performance 
aids  for  the  student.  These  overlays  include  a  path  of  circular  hoops  representing  the  desired 
endoscope  trajectory,  bullseye  targets  representing  the  desired  injection  sites,  and  text  labels 
identifying  anatomical  feature  landmarks. 

The  system  provides  voice  audio  feedback  representing  the  current  status  of  the  surgical  procedure, 
as  well  as  a  simulated  heartbeat  which  responds  to  certain  user  actions. 


Training  Tasks 

Students  using  the  system  are  instructed  to  perform  a  simulated  surgical  procedure  consisting  of 
three  subtasks:  navigation,  injection  and  dissection.  In  addition  to  archiving  the  frame-by-frame 
position  of  the  devices,  the  system  measures  the  time  required  to  perform  these  tasks,  as  well  as  the 
accuracy  with  which  they  are  performed,  to  generate  an  evaluation  of  the  student's  overall 
performance. 


Three  versions  of  the  procedure  were  developed  corresponding  to  three  skill  levels:  novice, 
intermediate,  and  advanced.  The  primary  differences  among  these  procedures  are  the  type  of 
geometric  model  used  and  the  presence  of  training  aids. 


The  novice  procedure  uses  a  simplified  abstract  model  of  the  anatomy  consisting  of  several  spheres 
inside  an  open  box.  Intermediate  and  advanced  procedures  use  the  more  complex  surface  sinus 
model  generated  by  OSC.  Novice  and  intermediate  procedures  use  the  training  aids  described 
above,  whereas  the  advanced  procedure  is  performed  without  benefit  of  these  aids  (to  more 
accurately  simulate  the  target  procedure). 
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1.2  Evaluation  Overview 

Evaluation  efforts  for  this  project  fall  into  two  general  categories: 

•  "Formative"  evaluation,  which  attempts  to  provide  design  specification  input  to  the 
development  team  during  the  development  process 

•  "Summative"  evaluation,  which  assesses  the  success  of  that  effort  by  formally  analyzing 
the  effectiveness  of  the  system 

Each  of  these  activities  is  discussed  in  detail  in  the  following  pages. 
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2  FORMATIVE  EVALUATION 

Throughout  the  development  of  the  ESS  simulator,  the  Human  Interface  Technology  Laboratory 
(HITL)  team  worked  in  close  collaboration  with  Dr.  Charles  Edmond,  staff  otolaryngologist  at 
Madigan  Army  Medical  Center  (MAMC)  to  perform  ongoing  formative  evaluation  and  make  design 
recommendations  to  the  development  teams  at  Lockheed  Martin,  OSC,  and  Immersion  Corporation. 

The  methods  of  approach  for  the  formative  evaluation  phase  included  the  following: 

1)  Endoscopic  video  analysis  to  determine  simulator  performance  requirements 

2)  Geometric  complexity  requirements  analysis 

3)  Prototype  anatomical  modeling 

4)  Development  of  spatial  awareness  aids,  interface  features  and  rendering  approaches 

5)  Development  of  a  prototype  simulator  with  an  integrated  expert  system  assistant 

6)  Development  of  a  surgical  training  curriculum  to  be  embedded  in  the  simulator 

7)  Survey  of  medical  experts  to  determine  feature  and  curriculum  priorities 

Dr.  Edmond  provided  the  core  of  domain  expertise  in  sinus  surgery.  After  a  few  weeks  of  study,  the 
rest  of  the  HITL  team  had  gained  a  basic  familiarity  with  sinus  anatomy  and  surgical  issues.  EXiring 
this  time  Dr.  Edmond  gained  familiarity  with  the  computer  graphics  tools  available  at  HITL  for  use 
in  prototyping  simulator  design  and  computer-assisted  surgery  applications. 

2.1  Design  Requirements  Analysis 

2.1.1  Frame  Rate  Requirements 

Method:  Representative  sequences  of  live  video  endoscopic  sinus  surgery  were  digitized  and 
manipulated.  These  sequences  were  selected  to  include  surgical  interaction  with  tissue  as  well  as  to 
highlight  early  and  late  operative  anatomy.  Image  degradation  due  to  the  scanning  process  itself 
was  minimal. 

We  then  re-recorded  the  scanned  video  at  the  following  frame  rates:  30  frames  per  second  (fps),  15 
fps,  10  fps,  5  fps,  2  fps,  and  1  fps.  These  recordings  were  reviewed  by  expert  surgeons  to  assess  the 
minimum  frame  rates  required  for  the  ESS  simulator. 

Results:  To  the  extent  possible,  30  fps  should  be  maintained.  For  surgical  dissection,  a  minimum  of 
15  fps  (preferably  30  fps)  should  be  maintained;  temporary  slowdowns  to  10  fps  during  deformation 
and  dissection  may  be  tolerable  for  certain  instruments  and  maneuvers  (e.g.,  side-biter,  but  not  sickle 
knife).  Frame  rates  lower  than  the  10  fps  minimum  may  cause  disorientation  in  navigation. 
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2.1.2  Geometric  Complexity  Requirements 

We  applied  the  following  guiding  principles  in  assessing  the  geometric  complexity  requirements: 

•  Visual  fidelity  requirements  are  task  driven. 

•  The  task  and  subtask  relevant  to  each  individual  varies  based  on  level  of  training  and 
experience. 

•  Three  categories  of  expertise  were  defined:  novice,  intermediate  and  expert.  Tasks  and 
subtasks  were  delineated  for  each  category,  and  fidelity  requirements  proposed. 

•  Accurate  anatomic  representation  (visual  realism)  and  3D  spatial  awareness  are  two  of  the 
most  critical  aspects  for  effective  cognitive  development.  This  was  felt  to  be  important 
across  all  levels  of  training  and  experience  for  endoscopic  sinus  surgery. 

•  The  development  of  psycho-motor  skills  necessary  for  successful  endoscopic  sinus 
surgery  may  require  less  visual  realism  and  more  3D  spatial  awareness. 

•  Low-level  visual  realism  might  be  less  distracting  to  the  novice,  and  therefore 
advantageous  for  both  developing  and  assessing  a  user’s  psycho-motor  skills. 

•  Varying  degrees  of  visual  fidelity  are  not  only  helpful,  but  necessary  to  develop  the 
cognitive  and  psycho-motor  skills  for  sinus  surgery. 

Methods:  To  help  assess  the  geometric  complexity  requirements  for  the  anatomical  models,  we 
generated  a  prototype  anatomical  model  using  the  following  technique: 

A  3D  triangular  mesh  was  generated  by  texture  mapping  an  actual  video  endoscopic  image  onto  a 
flat  mesh  surface  and  vertically  displacing  each  vertex  of  the  mesh  in  proportion  to  the  brightness  of 
texture  at  that  vertex.  We  then  varied  the  resolution  of  the  mesh  and  evaluated  the  quality  of  the 
resulting  images.  This  model  is  a  fairly  rough  approximation,  since  video  brightness  does  not 
precisely  correspond  to  geometric  distance  from  the  endoscope.  Specular  highlights  in  the  texture 
create  spike  artifacts  in  the  model.  These  were  eliminated  by  painting  over  the  highlights  in  the 
texture. 

Geometric  complexity  requirements  were  determined  by  visual  inspection  of  the  resulting  simulation 
mock-ups  by  the  project  domain  experts.  Evaluation  criteria  included: 

•  Ability  to  identify  features 

•  Subjective  evaluation  of  photo-realism 

•  Potential  for  simulator  miscues 

Results:  We  concluded  that  given  the  performance  of  the  rendering  system,  frame  rate  was  a  critical 
performance  requirement.  When  the  initial  versions  of  the  anatomy  model  were  delivered  from 
OSC,  the  frame  rate  did  not  meet  our  specification.  The  geometric  complexity  of  the  model  was 
reduced  until  the  specified  frame  rate  was  achieved.  The  geometric  complexity  requirement  was 
essentially  a  function  of  the  frame  rate  requirement  and  the  rendering  performance  of  the  hardware 
and  software  engines. 

The  anatomical  model  developed  for  the  ESS  simulator  adequately  served  its  function.  We  noted 
considerable  improvements  in  quality  and  efficiency  of  the  model  over  the  duration  of  our 
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evaluation.  Upon  each  release  of  a  new  model,  our  team  would  evaluate  the  model  for  rendering 
efficiency  and  fidelity.  Dr.  Edmond  would  suggest  modifications  to  the  model,  and  these  would  be 
executed  in  subsequent  releases.  The  addition  of  textural  cues  greatly  enhanced  the  apparent 
geometric  detail  without  sacrificing  rendering  speed. 

2.1.3  Prototype  Anatomical  Modeling 

Prototype  anatomical  virtual  models  were  developed  by  Edmond  and  Oppenheimer  using  the  Alias 
modeling  package.  The  purpose  of  these  models  was  to  provide  guidelines  in  the  design  of  the 
patient-specific  data-driven  models  produced  by  OSC,  as  well  as  to  determine  the  necessary 
rendering  and  interface  features  for  use  in  the  simulator. 

Methods:  The  geometry  of  these  prototypes  was  based  on  cadaveral  section  photographs  taken 
primarily  from  sinus  anatomy  atlases  and  surgery  textbooks  (Rohen  and  Yokochi,  1983;  Rice  and 
Schaefer,  1988).  In  one  case,  sections  were  scanned  into  the  computer,  and  mapped  onto  parallel 
image  planes.  These  image  planes  were  then  used  as  templates  for  drawing  surface  contours  on  the 
planes.  These  contours  were  then  lofted  into  surfaces.  Additional  contour  curves  were  added  and 
edited  as  needed,  based  on  Dr.  Edmonds's  observations. 

In  other  cases,  freeform  surface  contours  were  drawn  orthogonal  to  the  scanned  image  plane.  The 
scanned  image  was  then  projection  texture  mapped  onto  the  resulting  lofted  surface.  Although  less 
geometrically  accurate,  this  technique  enabled  photographic  texture  maps  to  be  used  in  the  final 
evaluation  model. 

Results:  These  prototype  models  provided  the  basis  for  experimenting  with  navigation  aids, 
interface  features  and  rendering  approaches  as  described  in  the  next  section. 

2.1.4  Interface  Features  and  Rendering  Approaches 

The  following  design  experiments  using  the  prototype  anatomical  models  resulted  in  recommen¬ 
dations  and  demonstrations  of  candidate  features  to  be  included  in  the  development  of  the  ESS 
simulator: 

•  Use  of  texture 

•  Use  of  transparent  or  wireframe  surfaces  to  reveal  obscured  anatomy 

•  Use  of  orientation  cues  to  assist  in  navigation 

•  Use  of  tubular  paths  of  hoops  as  a  navigation  aid 

•  Use  of  targets  as  an  injection  aid 

•  Use  of  crosshairs  overlaid  on  instruments  to  represent  orientation,  and  to  assess  distance 
from  anatomy 

•  Use  of  patient  face  model 

•  Anatomical  segmentation  and  interactive  selective  segment  display 

•  Displaying  endoscope  position  on  CT  scans  as  a  navigation  aid 

•  Integration  of  expert  system  training  aids 
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Texture  Mapping 

The  use  of  texture  mapping  offers  several  benefits.  Texture  maps  can  represent  detail  that  would  be 
too  expensive  to  model  as  polygonal  geometry.  Photographic  texture  maps  add  to  the  realism  of  the 
model. 

Texture  also  provides  additional  depth  cues.  If  two  parallel  surfaces  overlap  from  a  given 
perspective  view,  then  they  may  have  the  same  shading  coefficients  in  the  lighting  calculation. 
Without  texture  maps,  the  two  surfaces  would  be  rendered  with  the  same  pixel  values;  therefore  the 
boundary  between  them  would  be  indistinguishable.  By  placing  texture  maps  on  the  surfaces,  the 
discontinuity  between  the  surfaces  can  be  detected  more  easily,  as  a  consequence  of  texture 
discontinuity. 

In  conditions  of  extreme  ambient  lighting,  even  non-parallel  surfaces  may  have  similar  shading. 
Texture  mapping  can  reveal  subtle  differences  in  orientation  and  distance  from  observer.  In  "real 
life"  these  shading  ambiguities  are  also  noticeable  if  the  surfaces  are  of  uniform  texture.  Real-life 
objects  with  even  subtle  surface  textural  features  are  more  readily  discerned  than  smooth,  textureless 
objects. 

Methods:  Several  texture  mapping  methods  were  applied  to  the  models.  In  some  cases  an  attempt 
was  made  to  align  the  texture  with  the  underlying  model  geometry;  such  textures  included 
anatomical  features.  In  other  cases,  the  textures  were  closeup  details  of  tissue  type,  such  as  mucosa, 
and  did  not  include  anatomical  structures. 

Results:  On  the  basis  of  our  experiments,  we  recommended  the  use  of  texture  mapping  in  the  target 
simulator.  Since  the  polygonal  resolution  of  the  final  models  generated  from  the  Visible  Human 
database  is  as  fine  as  the  possible  textures  derived  from  the  same  data,  detail  texture  had  to  be  added 
algorithmically.  Such  texture,  although  not  photo-realistic,  does  add  to  the  realism  of  the  simulation 
as  well  as  improve  the  surface  boundary  visibility  as  described  above. 

Transparent  and  Wireframe  Surfaces 

By  rendering  outer  surfaces  transparently  or  in  wireframe,  the  system  can  display  otherwise 
obscured  portions  of  the  anatomy.  This  allows  students  to  view  anatomical  landmarks  prior  to 
revealing  them  through  dissection.  These  landmarks  may  include  anatomy  that  should  not  be 
dissected  (such  as  the  lamina,  optic  nerve,  or  skullbase).  Revealing  them  may  enhance  spatial 
awareness  training,  thereby  preventing  a  severe  surgical  error.  This  feature  is  not  currently 
implemented  in  the  target  simulator. 

Orientation  Cues 

By  displaying  a  world-stabilized  orthogonal  grid  or  crosshairs,  the  user  can  determine  the  orientation 
of  the  endoscope  with  respect  to  the  patient.  This  feature  is  not  currently  implemented  in  the  target 
simulator. 

Tubular  Path  of  Hoops 

By  displaying  a  tubular  surface  whose  axis  is  the  desired  trajectory  of  the  endoscope,  the  system  can 
provide  a  navigation  tool  to  the  user  during  surgical  simulation.  We  experimented  with  two 
different  rendering  modes  for  the  tube:  a  partially  transparent  surface  with  alpha-mapped 
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latitudinal  rings,  and  wireframe  latitudinal  hoops  oriented  orthogonal  to  the  axial  path.  The  target 
simulator  uses  the  latter  rendering  mode  (hoops). 

Injection  Targets 

Bullseye  targets  of  concentric  rings,  rendered  as  flat  shaded  polygons,  can  be  placed  on  the  anatomy 
at  points  of  injection.  These  targets  provide  cues  in  the  novice  and  intermediate  modes  of  the 
simulator. 

Crosshairs  Attached  to  the  Instrument 

By  attaching  polygonal  crosshairs  to  the  tip  of  the  virtual  instruments,  the  system  visually  represents 
the  relative  orientation  of  the  instrument  with  respect  to  the  endoscope.  If  a  grid  texture  is  mapped 
onto  the  crosshairs,  the  user  can  assess  the  distance  between  the  instrument  and  the  anatomy  by 
counting  the  grid  lines  between  the  instrument  tip  and  the  intersection  between  the  crosshair  and  the 
tissue  surface.  Grid  marks  deeper  than  the  tissue  surface  are  obscured  and  will  not  be  visible.  This 
feature  was  not  incorporated  into  the  current  implementation  of  the  simulator. 

Patient  Face  Model 

A  polygonal  surface  representation  of  the  patient  face  was  added  to  the  virtual  model.  This  face 
model  provides  the  student  with  a  position  and  orientation  cue,  as  well  as  providing  additional 
realism. 

Anatomical  Segmentation,  and  Interactive  Selective  Segment  Display 

The  prototype  anatomical  model  built  in  Alias  was  constructed  in  segmented  pieces,  corresponding 
to  recognizable  anatomical  features.  This  pre-segmentation  allows  one  to  selectively  display  certain 
parts  of  the  anatomy  as  well  as  to  highlight  selected  anatomical  features  at  run  time.  In  addition, 
virtual  tissue  segmentation  permits  collision  detection  with  procedurally  meaningful  objects.  The 
model  based  on  the  Visible  Human  dataset  required  an  editing  process  to  segment  the  surface 
anatomy. 

Displaying  Endoscope  Position  on  CT  Scans  as  a  Navigation  Aid 

As  originally  conceived,  the  user  interface  to  the  system  made  use  of  an  auxiliary  CT  view,  in 
addition  to  the  virtual  endoscopic  view  of  the  anatomy.  The  system  has  the  optional  capability  of 
updating  the  auxiliary  CT  image  view  with  a  crosshairs  indicator  of  the  current  endoscope  position. 
Although  not  used  by  the  subjects  in  the  current  evaluation  study,  this  feature  is  available  and  may 
be  useful  for  future  training  protocols. 

Prototype  Simulator  with  Expert  System  Assistant 

In  parallel  with  this  development  project,  HIT  Lab  researcher  Mark  Billinghurst  extended  his  work 
with  Jesus  Savage-Carmona  on  intelligent  multi-modal  environments  to  create  a  prototype  sinus 
surgery  simulator  with  an  integrated  expert  system  assistant  (Billinghurst  et  al.,  1996).  Working  with 
Oppenheimer  and  Dr.  Edmond,  Billinghurst  developed  a  system  which  incorporates  knowledge  of 
the  endoscopic  procedure  into  a  structured  rule  base,  interprets  the  user’s  multi-modal  inputs 
(currently  voice  and  virtual  endoscope  position)  and  interacts  with  the  user  dyadically.  While 
performing  the  simulated  procedure,  the  user  can  query  the  system  about  anatomy  and  the  specifics 
of  the  procedure,  asking  the  system  to  identify  features  or  demonstrate  maneuvers.  In  turn,  the 
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system  recognizes  the  user's  actions,  and  can  provide  vocal  and  visual  feedback,  as  well  as  warnings 
when  the  user  is  about  to  execute  a  dangerous  maneuver. 

Although  the  system  architecture  is  somewhat  different,  this  prototype  simulator  provided  ground 
work  demonstration  for  certain  features  of  the  target  surgical  simulator,  including  navigation 
through  the  sinus  cavity  from  an  endoscopic  viewpoint,  use  of  abstract  graphical  overlays  as  a 
navigation  aid,  and  embedding  of  surgical  task  sequences  into  simulator. 

2.1.5  Iterative  Testing 

In  addition  to  these  design  experiments  based  on  prototype  modeling,  the  HIT  Lab  served  as  a  test 
site  for  successive  versions  of  the  anatomical  model  and  the  simulation  system.  Having  the  target 
hardware  platform  (SGI  onyx)  in-house  made  iterative  evaluation  a  viable  and  useful  approach.  In 
particular,  the  proximity  of  the  lab  to  MAMC  made  it  relatively  easy  for  the  evaluation  team  to 
consult  with  Dr.  Edmond  regularly  on  simulator  features  and  system  performance. 

Upon  receiving  each  release  of  the  ESS  simulator,  the  HITL  team  would  compile  a  list  of  feature 
enhancements,  known  bugs  and  other  observations.  This  list  would  be  prioritized  by  Dr.  Edmond 
and  returned  to  Lockheed  Martin  for  inclusion  in  the  subsequent  release. 

Haptic  System  Emulator.  Instrument  tracking  and  force  feedback  displays  were  not  integrated  into 
the  earliest  test  releases  of  the  simulator.  A  menu-driven  "haptic  system  emulator"  (HSE)  provided  a 
graphical  user  interface  to  navigate  the  position  and  orientation  of  the  endoscope  and  instrument,  as 
well  as  to  select  the  instrument  type  and  control  the  open/ close  parameter.  The  HSE  consisted  of  up, 
down,  left,  and  right  buttons,  sliders  and  radio  buttons.  Although  not  driven  by  a  spatial  input 
device,  the  HSE  did  allow  for  precisely  controlled  navigation,  and  also  proved  useful  in  positioning 
navigation  aids  within  the  model. 

Although  initially  developed  as  a  prototype  testing  tool ,  the  HSE  may  have  some  commercial 
potential  as  well.  Customers  desiring  a  lower-cost  system  that  does  not  require  a  complete  6-degree  - 
of-freedom  interface  device  could  use  a  HSE  to  perform  the  surgical  tasks  (albeit  in  a  way  that  does 
not  capture  many  of  the  targeted  clinical  skills  of  the  Madigan  system).  In  this  case  the  up,  down, 
left,  and  right  buttons  could  be  replaced  with  a  mouse-based  interface  or  other  commercial  graphical 
input  device. 

2.2  Evolution  of  the  Training  Framework 

During  the  early  phases  of  the  project  we  examined  several  other  surgical  simulator  systems  to  look 
for  places  to  improve  the  state  of  the  art.  What  we  noticed  was  that  although  these  systems  were 
making  advances  in  anatomical  modeling  and  user  interaction,  they  were  missing  a  structured 
educational  component.  In  general  these  systems  simulated  a  specific  surgical  domain  or  task  only 
and  were  therefore  "orphaned"  experiences  in  the  educational  process. 

We  concluded  that  in  order  to  be  of  significant  educational  value,  one  had  to  not  only  embed  the 
simulator  in  the  existing  academic  curriculum,  but  also  develop  curriculum  within  the  simulation 
itself.  This  would  not  only  serve  the  educational  process  but  would  also  facilitate  our  evaluation  of 
the  educational  effectiveness  of  the  simulator. 

The  prevailing  paradigm  in  surgical  education  is  usually  summarized  as:  "See  One,  Do  One,  Teach 
One.”  Our  goal  was  to  improve  on  this  paradigm  by  taking  the  "See  One,  Simulate  Many,  Do  One" 
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approach  that  had  proved  effective  in  the  domain  of  flight  training.  To  achieve  this,  we  needed  to 
develop  a  curriculum  structure. 

2.2.1  Task  Analysis 

Working  with  Dr.  Edmond,  we  developed  a  taxonomy  of  ESS  simulation  objectives  and  simulator 
performance  and  interface  features.  Our  objective,  which  proved  far  too  demanding,  was  to  elicit 
estimates  from  experienced  ESS  surgeons  of  the  desirable  system  performance  requirements  for  each 
training  task.  This  approach  did,  however,  provide  us  with  a  framework  for  approaching  the  issues 
of  curriculum  design  in  this  domain. 

2.2.2  ESS  Domain  Expert  Survey 

Twelve  experienced  ESS  staff  surgeons  from  several  leading  otolaryngology  training  programs  were 
surveyed  by  Dr.  Edmond  to  assess  their  judgments  of  the  primary  simulation  requirements  for 
physicians  performing  ESS  procedures.  The  primary  objective  of  this  survey  was  to  determine  the 
curriculum  needs  and  perceived  importance  of  several  of  the  candidate  features  of  the  system  early 
in  its  development. 


Figure  2.  Mean  survey  response  ratings  (with  95%  error  bars)  of  11  training  and  operational  needs, 
for  three  hypothetical  target  groups  (novice,  intermediate,  and  experienced  ENT  physicians). 

These  domain  experts  were  asked  to  rate  the  relative  importance  (from  "critical  need"  to  "not 
important")  for  11  simulator  characteristics:  visual  realism,  spatial  awareness  training,  haptic 
interaction,  patient-specific  modeling,  psychomotor  training,  real-time  interactivity,  real-time  (intra- 
operational)  performance  aids,  standard  surgical  procedures,  complications,  advanced  techniques, 
and  pathophysiology. 

They  were  also  asked  to  assess  the  value  of  these  characteristics  for  target  subjects  at  three  levels  of 
ENT  experience:  novice  (i.e.,  junior  residents),  intermediate  (senior  residents),  and  expert 
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(experienced  ENT  surgeons).  Results  of  this  survey  are  summarized  in  Figure  2,  which  indicates  the 
mean  rating  for  each  characteristic  for  the  three  target  groups,  on  a  1-7  scale. 

As  can  be  seen  from  this  graph,  these  experts  deemed  "spatial  awareness"  the  most  crucial  training 
need  (of  those  presented  in  the  survey)  for  all  levels  of  subject  experience,  although  "advanced 
techniques"  were  seen  as  equally  critical  for  the  experienced  ESS  target  group.  With  the  exception  of 
"standard  procedures"  and  perhaps  "pathophysiology,"  all  requirements  were  seen  as  equally 
important  for  all  target  levels  or  more  important  with  increasing  ESS  experience. 

It  is  interesting  to  note  that  almost  all  of  these  characteristics  were  rated  on  the  "important"  end  of  the 
scale.  The  characteristics  deemed  least  important  for  simulation  were  "advanced  techniques"  and 
"patient-specific  modeling"  for  the  novice  group,  and  "standard  procedures"  for  the  advanced  group. 
In  addition,  "real-time  interactivity",  "haptic  interaction"  and  training  in  "complications"  were  seen  as 
of  only  moderate  importance  for  ESS  novices. 

Finally,  it  should  be  noted  that  these  survey  respondents  were  overwhelmingly  favorable  toward  the 
development  of  an  ESS  training  simulator.  While  one  or  two  expressed  skepticism  about  any 
technological  solutions,  most  respondents  noted  in  their  open-ended  comments  that  this  was  an 
interesting  and  useful  endeavor,  and  that  the  field  had  a  need  for  such  a  system. 

2.2.3  Simulator  Integration 

A  considerable  amount  of  effort  was  spent  during  the  development  phase  on  establishing  a 
"curriculum  wrapper"  for  the  simulator.  We  initially  envisioned  the  simulator  as  embedded  within  a 
multimedia  training  system  which  would  provide  a  meaningful  context  and  set  the  protocol  for  the 
trainee. 

The  development  team  elected  instead  to  integrate  relevant  aspects  of  the  emerging  ENT  curriculum 
into  the  simulator  itself  and  to  postpone  further  efforts  to  develop  a  total  curriculum  "package."  The 
reasons  for  this  included: 

•  Development  of  a  full-blown  ESS  curriculum  was  beyond  the  scope  of  this  phase  of  the 
project. 

•  It  became  apparent  that  a  staged  protocol  approach  was  necessary  to  make  the  simulator 
effective  as  a  training  context. 

•  Useful  techniques  emerged  (such  as  the  use  of  navigation  hoops  and  injection  targets) 
which  could  be  integrated  relatively  easily. 

The  final  product  of  the  formative  stage  of  the  project  was,  in  fact,  more  thoughtful  with  respect  to  an 
integrated  ENT  curriculum  than  had  originally  been  anticipated  and  appears  to  be  more  useful  as  a 
training  tool  than  other  virtual  reality  medical  simulators  which  have  emerged  in  recent  years.  Its 
validity  as  a  procedural  simulator  and  its  utility  for  training  are  evaluated  in  the  following  sections. 
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3  SUMMATIVE  EVALUATION 

After  delivery  and  installation  at  the  HIT  Lab  of  Version  1.2  of  the  system  by  the  Lockheed  Martin 
development  team,  the  evaluation  team  began  a  "shakedown"  of  the  system  and  testing  protocols 
with  the  three  targeted  subject  groups  (described  below).  Midway  through  this  phase  the  system 
was  relocated  to  MAMC  for  further  analysis  by  staff  and  resident  ENTs. 

The  primary  goals  of  this  phase  of  the  evaluation  effort  were  to: 

•  Validate  its  utility  as  an  ESS  training  environment 

•  Assess  the  usability  of  the  system 

•  Provide  additional  iterative  feedback  to  the  development  team 

The  methods  of  investigation  and  results  of  this  phase  are  described  below,  followed  by  a  set  of 
recommendations  for  further  system  development  and  evaluation  derived  from  these  findings.  In  a 
planned  follow-on  phase,  the  primary  focus  of  the  evaluation  effort  will  be  on  assessing  the  degree 
and  nature  of  any  transfer  of  this  training  to  the  real  ESS  operating  environment  (the  ultimate 
objective  of  the  simulator). 

3.1  Methods 

3.1.1  Subjects 

Subjects  were  solicited  from  three  distinct  groups:  (1)  non-physicians  with  general  intelligence  and 
psychomotor  abilities  roughly  comparable  to  the  average  otolaryngologist;  (2)  non-ENT  physicians 
from  a  variety  of  specialties;  and  (3)  ENTs  with  a  wide  range  of  ESS  experience.  We  focused  on  these 
three  groups  in  order  to  establish  a  baseline  and  asymptote  for  the  evaluation  of  the  efficacy  of  the 
simulator  in  training  otolaryngology  residents. 

The  first  two  groups  also  allowed  us  to  "shake  down"  the  system  and  research  protocols  without 
sacrificing  valuable  ENT  resident  subjects,  and  provided  baseline  scores  for  untrained/unfamiliar 
subjects.  Finally,  the  non-ENT  physician  group  could  provide  us  with  some  valuable  input  about  the 
extensibility  of  the  simulator  to  non-ENT  applications. 

Non-Physician  Group:  Twelve  volunteers  from  the  University  of  Washington  College  of 
Engineering  comprised  the  non-physician  group.  These  subjects  ranged  in  age  from  23  to  54,  and 
included  graduate  students,  professional  staff  and  faculty.  All  had  had  some  experience  with 
simulation  and  virtual  reality.  None  had  previously  used  an  endoscope  or  attended  medical  school. 

Non-ENT  Physician  Group:  Eight  University  of  Washington  MDs  from  specialties  other  than 
otolaryngology  provided  extensive  feedback  on  the  simulator  design  and  utility  for  other  medical 
and  surgical  tasks:  3  video  endoscopic  surgeons,  2  radiologists,  1  neurosurgeon,  1  cardiologist,  and  1 
anesthesiologist.  Four  of  them  also  provided  us  with  complete  trial  performance  data. 

ENT  Staff  and  Residents:  Twelve  staff  and  resident  ENTs  (1  female,  11  males)  from  MAMC  served 
as  subjects.  The  subjects  in  this  group  ranged  in  age  from  28  to  46,  with  a  mean  age  of  35.2  years, 
and  a  standard  deviation  of  6.15  years.  One  was  left-handed,  10  right-handed,  and  the  handedness 
of  one  was  unknown. 
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ENT  experience  for  this  group  broke  down  as  follows: 

•  4  staff  (with  an  average  of  five  years  of  training,  six  years  of  practice  and  more  than  100 
ESS  procedures  performed) 

•  3  R2s  (with  an  average  of  1-5  ESS  procedures  performed  or  observed) 

•  1  R3  (with  6-20  ESS  procedures  performed  or  observed) 

•  2  R4s  (with  an  average  of  21-100  ESS  procedures  performed  or  observed) 

•  2  R5s  (with  an  average  of  21-100  ESS  procedures  performed  or  observed) 

Eight  subjects  had  had  occasional  videogame  experience,  two  reported  playing  videogames  once, 
and  two  reported  never  playing  videogames.  Eleven  subjects  had  had  no  virtual  reality  experience, 
while  one  reported  occasional  VR  experience.  Ten  reported  having  no  other  simulator  experience, 
while  one  reported  one  simulation  experience  and  another  reported  "occasional"  simulation 
experience.  Because  of  the  low  incidence  of  prior  experience  with  these  systems,  these  factors  were 
not  evaluated  further  for  this  group. 
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ESS  performed 

Figure  3.  Frequency  distribution  of  the  reported  number  of  prior  ESS  procedures 
performed  by  subjects  in  the  ENT  group. 

Figure  3  shows  the  distribution  of  the  number  of  prior  ESS  procedures  performed  by  the  subjects  in 
this  group.  The  two  reporting  "1-5"  procedures  were  observational  only. 

3.1.2  Procedures 

All  subjects  proceeded  through  a  common  protocol  for  one  or  more  sessions,  involving: 

•  General  orientation  and  consent  form 

•  Pre-session  background  questionnaire  (Appendix  H) 
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•  Instructions  and  orientation  to  the  simulator  and  tasks 

•  One  or  more  proctored  simulation  trials  using  one  of  the  three  training  models 

•  Post-session  debriefing  and  questionnaire  (Appendix  I) 

All  trials  were  videotaped  for  later  analysis  and  "think-aloud"  comments.  Subjects  were  free  to 
terminate  the  session  at  any  time. 

Models 

Three  models  were  constructed  to  provide  for  a  sequentially  more  realistic  ESS  experience.  For  each 
model,  there  was  both  a  right-  and  left-side  version.  The  anatomy  was  built  for  the  right  nostril  and 
was  reflected  to  simulate  the  left  nostril.  The  nostril  (left  or  right)  with  which  the  subject  started  was 
determined  by  their  handedness. 

Model  1:  The  novice/abstract  model  consisted  of  only  the  skin  of  the  face  and  the  entrance  to  the 
nasal  cavity.  A  3D  grid  pattern  replaced  the  sinus  anatomy  to  provide  depth  of  field  during  the  three 
tasks:  navigation,  injection  and  dissection.  Training  aids  were  used  to  guide  the  subjects  through  the 
task.  Navigation  training  aids  consisted  of  virtual  hoops;  injection  training  aids  consisted  of  virtual 
targets  in  space. 

During  navigation,  the  subjects  maneuvered  the  endoscope  through  four  sets  of  virtual  hoops.  The 
paths  of  the  hoops  represented  three  passes  (sets  two  and  three  combined  are  one  pass)  commonly 
taken  before  the  surgery  begins  to  gain  familiarity  with  the  patient's  anatomy  and  to  allow  cleaning 
of  the  areas  of  interest. 

Injection  consisted  of  maneuvering  both  the  endoscope  and  instrumented  forceps  within  the 
environment  to  inject  five  targets  oriented  obliquely  in  space.  The  instrumented  forceps  controlled  a 
virtual  needle  for  this  task.  The  placement  of  the  targets  in  space  reflected  the  common  areas  of 
injection  of  a  vasoconstrictor  during  a  limited  ethmoidectomy. 

During  dissection,  the  subjects  were  also  required  to  use  both  the  endoscope  and  the  instrumented 
forceps.  The  task  consisted  of  dissecting  each  of  a  series  of  virtual  spheres  with  pre-selected  virtual 
tools.  The  instrumented  forceps  represented  each  of  the  tools  most  commonly  used  in  the  procedure: 
freer,  needle,  bent  needle,  sickle  knife,  microdebrider,  suction,  straight-biting  forceps,  up-biting 
forceps,  left-biting  forceps  and  right-biting  forceps. 

Navigation  through  the  four  sets  of  hoops,  injection  of  the  five  targets  and  dissection  of  each  of  the 
spheres  was  required  for  a  complete  score.  Digitized  voice  audio  cues  were  given  for  each  hoop 
negotiated  in  navigation,  for  the  percentage  of  each  target  injected,  for  percentages  completed  of  each 
sphere  during  dissection,  and  for  final  completion  of  each  subtask. 

Model  2:  The  intermediate  model  was  composed  of  the  navigation  and  injection  training  aids  from 
Model  1,  overlaid  within  a  virtual  anatomical  model  of  the  sinus  cavity.  Injection  and  dissection 
followed  the  protocol  for  a  limited  ethmoidectomy:  injection  of  the  inferior/ anterior  middle 
turbinate,  superior  root  of  the  middle  turbinate  and  the  lateral  nasal  wall,  followed  by  dissection  of 
the  uncinate  process,  bulla  ethmoidalis  and  posterior  ethmoid  cells.  Labels  were  added  for  all 
anatomical  structures  with  which  the  subject  interacted. 

Navigation  through  the  four  sets  of  hoops,  injection  of  the  five  targets,  medialization  of  the  middle 
turbinate,  dissection  of  the  three  anatomical  structures,  removal  of  two  bone  fragments  placed  in  the 
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uncinate  process  and  removal  of  three  bone  fragments  placed  in  the  bulla  ethmoidalis  were  required 
for  a  complete  score.  The  widening  of  the  maxillary  ostium  was  not  included  in  the  procedure  due  to 
the  inability  to  dissect  enough  of  the  lateral  part  of  the  uncinate  process  for  realistic  viewing  of  the 
ostium.  Audio  cues  were  given  for  each  hoop  negotiated  in  navigation,  for  each  target  hit  in 
injection,  for  each  bone  fragment  removed  and  for  percentages  completed  for  each  anatomical 
structure  in  dissection. 

Model  3:  The  advanced  model  was  composed  of  an  anatomical  model  only.  Subjects  were  expected 
to  perform  the  three  tasks  without  the  training  aids  and  to  follow  the  protocol  for  a  limited  ethmoid- 
ectomy.  Three  polyps  were  added  superior/ anterior  to  the  bulla  ethmoidalis. 

During  navigation,  the  subject  was  required  to  perform  the  three  passes  in  the  same  order  as  in 
Models  1  and  2:  inferior  pass  along  the  floor  of  the  nose  to  the  nasopharynx,  followed  by  a  more 
superior  pass  medial  to  the  middle  turbinate  towards  the  upper  aspect  of  the  nasopharynx  and 
sphenoid  ostium,  then  rolling  under  the  middle  turbinate  to  inspect  the  ostial  meatal  complex,  and 
finally  the  superior  pass  medial  to  the  root  of  the  middle  turbinate  towards  the  sphenoethmoidal 
recess. 

During  injection,  the  subject  was  cued  only  by  the  amount  of  blanching  (whitening)  of  the  virtual 
tissue  as  to  whether  more  vasoconstrictor  was  needed. 

Dissection  followed  the  protocol  for  a  limited  ethmoidectomy.  Navigation  through  the  three  passes, 
injection  of  the  areas  of  interest,  medialization  of  the  middle  turbinate,  dissection  of  the  three 
anatomical  structures,  dissection  of  the  three  polyps,  removal  of  two  bone  fragments  placed  in  the 
uncinate  process  and  removal  of  three  bone  fragments  placed  in  the  bulla  ethmoidalis  were  required 
for  a  complete  score.  Audio  cues  were  given  only  at  the  end  of  each  of  the  navigation  passes,  for 
removal  of  bone  fragments  and  for  percentages  completed  in  injection  and  dissection. 

Proctoring 

During  each  trial,  a  proctor  with  knowledge  of  the  procedure  was  present;  proctoring  for  each  of  the 
groups  varied  according  to  their  familiarity  with  the  task.  To  assure  that  all  records  of  the  trial  and 
all  subject  comments  were  noted,  a  second  proctor  was  usually  present.  One  proctor  was  designated 
as  an  instructional  proctor  who  would  introduce  the  subject  to  the  simulator  and  answer  any 
questions  during  the  trial.  The  second  proctor  would  manage  the  forms,  records  and  loading  of  the 
trials  for  the  subject. 

Non-Physician  Group 

Each  subject  was  initially  introduced  to  the  simulator  and  informed  that  we  were  evaluating  the 
simulator  as  a  possible  trainer  for  residents  in  Otolaryngology.  A  summary  of  the  reasons  for  this 
type  of  surgery  was  given  along  with  a  brief  introduction  to  the  anatomical  structures  and  their 
locations  in  the  sinus  cavity  to  give  the  subjects  a  feel  for  the  dimensions  of  the  area  in  which  they 
would  be  working. 

Subjects  were  then  introduced  to  the  instrumented  endoscope  and  informed  of  their  ability  to  rotate 
the  image  axially  by  rotating  the  shaft  of  the  endoscope.  They  were  then  introduced  to  the 
instrumented  forceps  and  informed  that  the  forceps  would  be  simulating  the  virtual  needle  for 
injection  and  all  dissection  tools.  The  mechanics  of  the  forceps  (open  and  closed)  were  described  as 
simulating  the  plunger  for  the  syringe  during  injection  and  opening  and  closing  the  jaws  of  the 
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dissection  tools.  They  were  then  shown  how  the  instrumented  forceps  would  be  positioned  in  the 
opposite  nostril  until  the  beginning  of  injection. 

Subjects  were  given  a  brief  verbal  description  of  the  three  tasks  for  Model  1  (navigation,  injection  and 
dissection)  and  what  would  be  required  of  them  during  the  trial.  They  were  informed  that  their 
introduction  to  Model  1  would  be  broken  up  across  the  three  subtasks.  A  videotape  of  Dr.  Edmond 
performing  the  trial  was  then  started,  during  which  the  subject  was  allowed  to  step  up  to  the 
mannequin  and  become  familiar  with  the  instrumentation,  while  the  proctor  described  the  task  in 
more  detail.  Subjects  were  encouraged  to  ask  any  questions  and  "think  aloud"  during  the  entire 
procedure. 

Following  instructions  and  video  for  navigation,  the  subject  independently  performed  the  navigation 
task.  The  trial  was  then  paused,  proctoring  and  video  for  injection  was  given,  and  the  injection  task 
was  performed.  At  that  time  the  proctor  would  pull  the  instrumented  forceps  across  the  columella 
nasi  into  the  nostril  being  used  and  place  a  plug  in  the  original  nostril  to  inhibit  re-crossing  of  the 
columella  nasi  by  the  instrumented  forceps  during  the  remainder  of  the  trial.  After  injection,  the  trial 
was  then  paused,  instructions  and  video  for  dissection  were  provided,  and  the  dissection  task  was 
performed. 

Model  1  was  the  only  model  where  this  process  was  used  and  it  was  used  only  on  their  initial 
introduction  to  the  simulator.  On  subsequent  trials  of  Model  1,  they  were  given  verbal  proctoring 
instructions  only,  and  allowed  to  perform  the  task.  Those  who  were  introduced  to  Model  2  did  not 
require  breaking  the  trial  into  subtasks. 

Progression  to  Model  2  was  based  on  performance  on  Model  1.  An  average  score  of  54%  was 
required  of  the  subjects  on  Model  1  before  progressing  to  Model  2.  Since  subjects  in  this  group  had 
no  familiarity  with  the  procedure  or  with  paranasal  sinus  anatomy.  Model  2  required  extensive 
proctoring  instructions  on  locations  and  anatomy  to  dissect  during  the  procedure.  No  subjects  from 
this  group  were  run  through  Model  3  because  of  their  inability  to  achieve  adequate  proficiency  in 
Model  2  during  the  time  course  of  the  study. 

Non-ENT  Physician  Group 

Again,  each  subject  was  initially  introduced  to  the  simulator  and  informed  that  we  were  evaluating 
the  simulator  as  a  possible  trainer  for  residents  in  Otolaryngology.  They  were  encouraged  to  think  of 
ways  in  which  this  type  of  simulator  could  be  used  in  their  own  fields.  They  were  then  introduced  to 
the  virtual  endoscope,  informed  of  their  ability  to  rotate  the  image  axially  by  rotating  the  shaft  of  the 
endoscope,  and  of  the  availability  of  30°  and  70°  scopes  which  could  be  swapped  for  the  zero  degree 
scope  they  would  initially  be  given.  The  optics  of  the  30°  and  70°  scopes  were  explained  where 
necessary. 

Subjects  were  then  introduced  to  the  instrumented  forceps  and  informed  that  the  instrumented 
forceps  would  simulate  the  virtual  needle  for  injection  and  all  dissection  tools.  The  mechanics  of  the 
instrumented  forceps  (open  and  closed)  were  described  as  simulating  the  plunger  for  the  syringe 
during  injection  and  opening  and  closing  the  jaws  of  the  dissection  tools.  They  were  then  informed  of 
how  the  instrumented  forceps  would  be  positioned  in  the  nostril  opposite  the  one  used  in  the  trial, 
until  the  beginning  of  injection. 

The  subjects  were  then  given  a  brief  verbal  description  of  the  three  tasks  of  Model  1  (navigation, 
injection  and  dissection)  and  what  would  be  required  of  them  during  the  trial.  A  video  of  Dr. 
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Edmond  performing  the  trial  was  then  started,  while  the  proctor  continued  to  describe  the  subtasks 
in  more  detail  and  what  was  required  for  completion  of  the  trial.  During  this  time  the  subject  was 
allowed  to  step  up  to  the  mannequin  and  become  familiar  with  the  instrumentation.  The  'blood 
effects  scope"  (scope  becoming  opaque  within  a  set  time  interval)  was  shown  in  the  video  and 
reasons  for  it  were  described  by  the  proctor,  along  with  how  to  relieve  the  problem  by  wiping  the 
"scope"  on  the  foam  pad  located  on  the  mannequin's  forehead. 

The  subject  was  encouraged  to  ask  any  questions  and  "think  aloud"  during  the  procedure.  This 
process  was  repeated  before  their  first  introduction  to  Model  2,  with  the  appropriate,  model-specific 
changes.  On  subsequent  trials  of  Model  1  and  Model  2  they  were  given  verbal  proctoring 
instructions  only  before  performing  the  task. 

The  subject's  familiarity  with  the  procedure  determined  the  need  for  further  instruction  by  the 
proctor.  On  average,  this  group  had  no  familiarity  with  the  procedure,  but  all  had  prior  introduction 
to  anatomy  (during  medical  school)  and  an  understanding  of  the  reasons  for  the  procedure. 
Progression  to  Model  2  for  this  group  was  based  on  performance  on  Model  1;  an  average  score  of 
69%  was  required  on  Model  1  before  progressing  to  Model  2.  Proctoring  for  Model  2  required 
instructions  on  locations  and  anatomy  to  dissect  during  the  procedure.  Training  aids  and  video  of 
the  procedure  provided  adequate  introduction  to  the  task  for  Model  l's  entirety  and  for  Model  2’s 
navigation  and  injection  tasks.  No  subjects  from  this  group  were  run  through  Model  3,  due 
primarily  to  the  limited  availability  of  these  MD  subjects. 

ENT  Staff  and  Residents 

Again,  each  subject  was  initially  introduced  to  the  simulator  and  informed  that  we  were  evaluating  it 
as  a  possible  trainer  for  residents  in  Otolaryngology.  They  were  introduced  to  the  virtual  endoscope, 
informed  of  their  ability  to  rotate  the  image  axially  by  rotating  the  shaft  of  the  endoscope,  and  of  the 
availability  of  30°  and  70°  scopes  which  could  be  swapped  for  the  zero  degree  scope  they  would 
initially  be  given. 

Subjects  were  then  introduced  to  the  instrumented  forceps  and  informed  that  they  would  simulate 
the  virtual  needle  for  injection  and  all  dissection  tools.  The  mechanics  of  the  instrumented  forceps 
(open  and  closed)  were  described  as  simulating  the  plunger  for  the  syringe  during  injection  and 
opening  and  closing  the  jaws  of  the  dissection  tools.  They  were  then  informed  of  how  the 
instrumented  forceps  would  be  positioned  in  the  nostril  opposite  the  one  used  in  the  trial  until  the 
beginning  of  injection. 

The  subjects  were  then  given  a  brief  verbal  description  of  the  three  tasks  of  Model  1  and  what  would 
be  required  of  them  during  the  trial.  The  video  of  Dr.  Edmond  performing  the  trial  was  then  started, 
while  the  proctor  continued  to  describe  the  subtasks  in  more  detail  and  what  was  required  for 
completion  of  the  trial.  During  this  time,  the  subject  was  allowed  to  step  up  to  the  mannequin  and 
become  familiar  with  the  instrumentation. 

The  "blood  effects  scope"  was  shown  in  the  video  and  reasons  for  it  were  described  by  the  proctor, 
along  with  how  to  relieve  the  problem  by  wiping  the  scope  on  the  foam  pad  located  on  the 
mannequin's  forehead.  Subjects  were  encouraged  to  ask  any  questions  and  "think  aloud"  during  the 
procedure.  On  subsequent  trials  they  were  given  verbal  proctoring  instructions  only  (with  the 
appropriate,  model-specific  changes),  and  allowed  to  perform  the  task. 
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The  subject's  familiarity  with  the  procedure  determined  the  need  for  further  instruction  by  the 
proctor.  Typically,  the  staff  otolaryngologists  needed  no  further  instruction  on  the  procedure  for  the 
remainder  of  the  session,  except  for  the  need  to  be  shown  the  active  areas  for  dissection  of  the 
uncinate  process,  bulla  ethmoidalis  and  posterior  ethmoid  cells  within  the  virtual  model.  In  general, 
the  staff  ENT  subjects  ran  through  all  three  model  levels. 

Similarly,  the  ENT  residents  needed  no  further  instruction  for  Model  1.  For  Model  2,  however,  more 
detailed  instructions  were  sometimes  needed  during  dissection  on  what  anatomy  to  dissect  and 
where  the  active  dissection  areas  were  located  within  the  anatomy.  In  Model  3  more  detailed 
instructions  were  also  needed  during  navigation  on  the  order  of  passes  to  perform  and  where  the 
active  dissection  areas  were  located  within  the  anatomy.  All  residents  were  run  through  Model  1- 
right.  Model  2-right,  Model  3-right,  Model  3-left  and  Model  2-left. 

User  Performance  Criteria 

The  scoring  algorithm  for  Version  1.2  takes  into  consideration  three  major  performance  measures  for 
endoscopic  sinus  surgery:  accuracy,  completeness  and  time.  The  overall  trial  score  is  calculated  as: 

(navigation  score  +  injection  score  +  dissection  score)  /  3 

Subtask  scores  are  calculated  as: 

subtask  score  =  accuracy*optimal  time  /  completed  time 

The  scoring  algorithm  used  for  the  evaluation  of  this  phase  of  the  project  took  into  account  the  most 
important  skills  needed  to  perform  sinus  surgery,  as  well  as  the  level  of  difficulty  of  the  model.  In  an 
attempt  to  normalize  across  model  conditions,  the  following  optimal  times  (in  seconds)  were  used  in 
this  scoring  equation: 

Navigation:  advanced=97,  intermediate=97,  novice=97 

Injection:  advanced=90,  intermediated  65,  novice=90 

Dissection:  advanced=815,  intermediate=635,  novice=165 

These  times  were  derived  from  the  approximate  performance  times  for  the  project’s  lead  otolaryn¬ 
gology  advisers.  If  a  subject  performed  the  subtask  in  less  than  the  optimal  time,  then  their  score 
was  equal  to  their  accuracy  on  that  subtask. 

Navigation  accuracy  was  based  on  the  percentage  of  hoops  negotiated  (with  the  hoops  in  the 
advanced  model  rendered  invisibly).  Injection  accuracy  scores  were  based  on  the  ratio  of  the 
percentage  of  each  target  injected  to  a  criterion  percentage  for  that  object:  100%  for  the  graphical 
targets,  25%  for  the  middle  turbinate,  and  10%  for  the  nasal  wall.  Similarly,  dissection  accuracy  was 
based  on  a  criterion  dissection  percentage  for  each  dissectable  object  (markers,  polyps,  uncinate, 
ethmoid  cells,  ethmoid  bulla,  maxillary  ostium,  and  bone  fragments). 

3.2  Results 

3.2.1  Non-Physician  Evaluation 

Although  not  the  target  domain  audience  for  this  simulator,  the  non-physician  group  gave  us  an 
opportunity  to  look  at  the  baseline  difficulty  of  the  simulator  for  domain-naive  users  and  to  further 
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work  out  requirements  for  instructional  presentation,  proctoring,  and  pacing  of  the  "curriculum" 
(after  initial  "shake  down"  and  protocol  development  by  the  project  staff).  Since  the  novice  model 
did  not  require  interaction  with  the  complex  paranasal  sinus  anatomy  or  detailed  knowledge  of  the 
surgical  procedure,  it  was  felt  that  this  group  should  be  able  to  complete  those  trials. 

The  inherent  difficulty  of  the  task  for  inexperienced  users  is  evidenced  by  subject  attrition  and  by 
their  initial  trial  scores.  Indeed,  complete  trial  data  are  available  for  only  nine  of  the  12  subjects  in 
this  group.  Of  the  three  lost  to  attrition,  one  terminated  the  session  due  to  disorientation  and 
discomfort  with  visual  interface,  and  two  were  unable  to  complete  even  the  navigation  task  during 
the  time  period  available  for  testing.  The  primary  difficulty  observed  with  these  three  subjects  was 
an  inability  to  adapt  to  the  psychomotor  demands  of  the  interface;  that  is,  they  were  not  able  to 
acquire  the  basic  skill  necessary  to  control  the  positioning  of  the  virtual  endoscope  within  the  model. 

Of  the  nine  remaining  subjects  in  this  group,  five  had  had  considerable  experience  with  videogames 
and  three  with  commercial  flight  simulators.  Thus  the  trials  scores  for  this  group  may  represent  the 
high  end  of  non-physician  users  and  should  not  be  taken  as  a  sample  from  the  general  non-physician 
population. 

Novice  Model  Performance 

Still,  despite  attrition  of  the  low-end  performers  and  the  unusual  prior  experience  of  the  remaining 
subjects,  the  non-physician  group  performed  significantly  worse  than  the  ENT  group  on  their  initial 
novice  level  trials  (as  can  be  seen  graphically  in  a  later  section). 

Unpaired  student’s  t  comparisons  with  the  ENT  group  revealed  significantly: 

lower  overall  trial  scores  (non-physician  mean  =  45.9,  ENT  mean  =  65.1,  t  =  2.87,  p  =  .0098), 

longer  overall  trial  times  (non-physician  mean  =  1012  sec,  ENT  mean  =  692  sec,  t  =  2.14,  p  = 
.0458), 

lower  navigation  scores  (non-physician  mean  =  47.1,  ENT  mean  =  69.4,  t  =  2.39,  p  =  .0276), 

longer  navigation  times  (non-physician  mean  =  239  sec,  ENT  mean  =  146  sec,  t  =  2.52,  p  = 
.0209),  and 

lower  injection  scores  (non-physician  mean  =  55.3,  ENT  mean  =  81.5,  t  =  2.46,  p  =  .0235) 

Injection  times,  while  not  significantly  longer,  did  approach  significance  (non-physician  mean  =  257 
sec,  ENT  mean  =  117  sec,  t  =  1.96,  p  =  .0649).  Surprisingly,  while  dissection  scores  and  dissection 
times  were  slightly  worse  for  the  non-physician  group,  these  differences  were  not  significant.  This 
relatively  poor  performance  by  the  ENT  group  on  their  initial  dissection  trial  is  discussed  further 
below. 

Practice  Effect 

Six  of  the  12  subjects  in  this  group  had  complete  trial  scores  for  at  least  two  novice  trials.  Students'  t 
comparisons  of  their  first  and  second  novice  trials  showed  that  their  second  trial  resulted  in 
significantly: 

Higher  overall  trial  scores  (trial  1  mean  =  46.7,  trial  2  mean  =  69.5,  t  =  2.33,  p  =  .0423), 

Higher  injection  scores  (trial  1  mean  =  53.8,  trial  2  mean  =  86.7,  t  =  2.22,  p  =  .0500), 
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Shorter  dissection  times  (trial  1  mean  =  492  sec,  trial  2  mean  =  328  sec,  t  =  2.39,  p  =  .0382),  and 

Higher  dissection  scores  (trial  1  mean  =  35.8,  trial  2  mean  =  51.8,  t  =  2.55,  p  =  .0287). 

Total  trial  times,  navigation  times  and  scores,  and  injection  times  all  changed  in  the  expected 
direction  (that  is,  they  improved  across  trials),  but  none  of  these  improvements  was  statistically 
significant.  It  appears  that,  for  this  non-physician  group,  the  primary  challenge  may  have  been 
mastering  the  endoscope  positioning. 

Intermediate  Model  Performance 

First-time  intermediate  level  scores  were  acquired  for  only  3  of  the  subjects  in  the  non-physician 
group.  Despite  this  small  N,  unpaired  student's  t  comparisons  with  the  ENT  group  revealed 
significantly 

Lower  overall  trial  scores  (non-physician  mean  =  53.7,  ENT  mean  =  76.7,  t  =  3.53,  p  =  .0041), 

Longer  overall  injection  times  (non-physician  mean  =  359  sec,  ENT  mean  =  132  sec,  t  =  6.91,  p 
=  .0001),  and 

Lower  injection  scores  (non-physician  mean  =  47.1,  ENT  mean  =  69.4,  t  =  2.39,  p  =  .0276). 

Overall  trial  times,  while  not  significantly  longer,  did  approach  significance  (non-physician  mean  = 
1532  sec,  ENT  mean  =  1215  sec,  t  =  1.79,  p  =  .098),  as  did  navigation  times  (non-physician  mean  =  199 
sec,  ENT  mean  =  138  sec,  t  =  2.08,  p  =  .0592)  and  navigation  scores  (non-physician  mean  =  48.0,  ENT 
mean  =  73.7,  t  =  1.99,  p  =  .0698).  Again,  no  difference  was  found  between  the  two  groups  in  initial 
dissection  times  and  dissection  scores  on  the  intermediate  model. 

The  experiential  advantage  of  the  ENTs  is  perhaps  most  telling  in  their  superior  injection  task 
performance.  As  will  be  seen  below  in  examining  trends  within  the  ENT  group,  ability  to  perform 
efficient  injections  may  by  the  hallmark  of  the  accomplished  surgeon,  much  more  so  than  the  ability 
to  perform  efficient  dissection  (at  least  as  indicated  by  simulator  performance). 

3.2.2  Non-ENT  Physician  Evaluation 

The  primary  objectives  of  the  non-ENT  physician  evaluations  were  to  provide  a  medically  trained 
comparison  and  "shakedown"  group  and  to  explore  the  perspectives  of  other  medical  specialties 
regarding  additional  potential  applications  for  this  sort  of  simulation  training.  In  particular,  we  were 
interested  in  applications  which  might  make  use  of  the  integrated  approach  and  specific  components 
incorporated  into  the  Madigan  ESS  simulator. 

Novice  Model  Performance 

Novice  performance  measures  for  the  non-ENT  MD  group  were  very  similar  to  those  for  the 
experienced  ENT  group  (as  summarized  graphically  below  in  the  ENT  analysis  section).  While  there 
were  no  statistically  significant  differences  between  these  two  groups,  it  should  be  noted  that  the 
mean  differences  for  all  of  the  performance  measures  were  in  the  direction  we  had  expected  (that  is, 
the  mean  performance  measures  were  consistently  better  for  the  ENT  group). 

This  finding  may  be  due  to  the  considerably  broad  prior  medical  and  interface  experience  for  all  of 
the  subjects  in  this  group.  Most  of  the  subjects  in  this  group  have  been  involved  over  the  years  in 
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developing  or  testing  novel  interface  devices  for  medical  tasks,  and  several  had  previous  experience 
test-driving  similar  medical  simulators. 

Perhaps  the  greatest  mitigating  factor,  however,  is  the  extensive  experience  that  two  of  these  four 
subjects  had  acquired  as  videoendoscopic  surgeons.  This  experience  provided  them  with  a 
background  comparable  to  the  ENT  group  in  basic  endoscopic  psychomotor  skills,  as  well  as  in 
general  procedural  knowledge  and  confidence. 

Although  there  were  no  statistically  significant  differences  in  any  performance  measures  between  the 
non-ENT  MDs  and  the  non-physician  subjects,  it  was  apparent  that  they  were  much  more 
comfortable  with  the  experimental  task  requirements.  11118  finding  (and  the  nature  of  the  questions 
asked  during  the  session)  suggests  the  value  of  a  general  medical  perspective  in  assuring  confidence 
with  this  sort  of  procedural  simulator. 

Intermediate  Model  Performance 

Three  of  the  non-ENT  MD  subjects  also  tried  the  intermediate  model.  Comparison  of  the  non-ENT 
MDs  with  the  ENTs  on  their  first  trial  on  the  intermediate  model  reveals  an  interesting  finding. 

While  there  was  no  significant  difference  between  the  groups  in  overall  score  and  trial  time,  the  non- 
ENTs  did  notably  better  than  the  ENTs  on  the  dissection  subtask.  Dissection  scores  were 
significantly  better  for  the  non-ENTs  (non-ENT  mean  =  84.0,  ENT  mean  =  63.1,  t  =  2.439,  p  =  .0312), 
which  appears  to  be  due  primarily  to  their  faster  times  on  that  task  (non-ENT  mean  =  662  sec,  ENT 
mean  =  880  sec,  t  =  1.99,  p  =  .0692),  a  non-significant  but  clearly  suggestive  finding. 

Looking  at  individual  scores,  we  note  that  this  effect  is  due  to  the  relatively  fast  dissection  times  for 
the  two  videoendoscopic  surgeons.  While  we  would  expect  that  the  ENT  group  would  be  superior 
to  all  other  groups  on  this  task,  the  extensive  background  in  similar  procedures  appears  to  have 
prepared  the  general  surgeons  well  for  this  task. 

Related  Medical  and  Surgical  Applications 

Response  by  the  non-ENT  physician  group  was  uniformly  positive.  Each  of  them  suggested 
additional  applications,  both  within  and  outside  of  their  particular  specialties,  that  would  be 
amenable  to  a  similar  simulation  approach.  These  suggested  applications  included: 

•  Neurosurgery:  approaching  the  pituitary  gland  through  the  sphenoid  sinus 

•  Anesthesiology:  intubation,  bronchioscopy 

•  Videoendoscopic  surgery:  cholecystectomy,  bladder,  joints,  throat,  chest  procedures 

•  Cardiology:  cannulization,  thoracoscopic  surgery 

•  Gastroenterology:  endoscopic  procedures 

•  Radiology:  fluoroscopy,  intrusive  ultrasound  procedures  (e.g.,  transesophageal 
echocardiography) 

In  general,  procedures  that  use  endoscopes  or  probes  to  explore  internal  structure  and  to  perform 
manual  procedures  should  be  amenable  to  this  simulation  approach.  Several  of  the  physicians  in  this 
group  indicated  that  they  would  like  to  see  work  proceed  in  that  direction  and  that  there  would  be 
support  for  this  development  from  the  leaders  within  their  specialties. 
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3.2.3  Staff  and  Resident  ENT  Evaluations 

Simulator  performance  measures  were  acquired  on  53  separate  trials  for  the  12  ENT  subjects  between 
5/13/97  and  8/12/97  (23  trials  run  at  HITL,  and  30  trials  run  at  MAMC).  The  number  of  trials 
acquired  for  each  subject  ranged  from  2  to  12,  with  all  subjects  being  tested  (minimally)  on  both  the 
novice  and  the  intermediate  model. 

Initial  Novice  Model  Performance 

Figure  4  shows  the  distribution  of  trial  scores  on  the  initial  novice  trials  for  the  12  ENT  subjects. 

To  examine  the  degree  to  which  prior  OR  experience  might  contribute  to  these  initial  score  differ¬ 
ences,  we  calculated  the  Pearson  product-moment  correlation  coefficients  ("r")  across  the  primary 
performance  measures  (trial  time  and  trial  score)  and  the  primary  experience  measures  (age,  years  of 
ESS  training,  and  approximate  number  of  actual  ESS  procedures  performed.  A  matrix  of  these 
correlations  appears  in  Table  1  (critical  r  =  .6021  for  probability  p  <  .05,  degrees  of  freedom  df  =  9). 
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Figure  4.  Frequency  distribution  of  overall  scores  on  initial  trial 
on  novice  model  by  subjects  in  ENT  group. 

As  can  be  seen,  the  best  predictor  of  novice  trial  time  from  among  the  surgical  experience  measures  is 
the  number  of  actual  ESS  cases  performed  (r  =  -.652),  with  trial  time  decreasing  significantly  with 
surgical  experience.  For  trial  score,  the  findings  are  less  conclusive,  although  the  best  experiential 
predictor  is  again  the  number  of  ESS  cases  performed  (r  =  .526).  While  age  is  naturally  correlated 
with  ESS  experience  and  training,  age  alone  does  not  appear  to  be  a  significant  predictor  of  simulator 
performance  for  these  ENT  subjects. 
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Table  1.  Correlation  matrix  showing  relationships  among  ESS  experience  measures 
and  overall  performance  measures  on  the  novice  model  for  12  ENT  subjects. 


Snore 

FSS  perf 

Years  tr 

Age _ 

Trialtime 

Score 

1 

ESS  performed 

.526 

1 

Years_training 

.38 

.923 

1 

Age 

.236 

.662 

.713 

1 

Trialtime 

-.966 

-.652 

-.514 

-.33 

1 

Note:  1  case  deleted  with  missing  values. 

In  general,  we  see  that  trial  score  on  the  initial  novice  simulator  trial  is  indeed  positively  correlated 
with  prior  ESS  experience.  This  finding  suggests  that  the  simulator  provides  a  valid  reflection  of  the 
skills  acquired  in  ESS  procedures. 

Subtask  Performance  on  Initial  Novice  Trial 

Not  surprisingly,  overall  trial  scores  are,  with  the  sole  exception  of  injection  time,  significantly 
correlated  with  subtask  scores  and  times,  as  shown  in  Table  2  (critical  r  =  .5760  for  p  <  .05,  df  10). 
Overall  trial  times  are  similarly  related  to  subtask  performance  measures,  although  by  far  the 
strongest  predictor  of  overall  trial  time  is  performance  on  the  dissection  task  (r  =  .897  for  dissection 
time  and  -.918  for  dissection  score). 

Table  2.  Correlation  matrix  showing  relationships  among  ESS  procedures  previously  performed 
and  subtask  performance  measures  on  novice  model  for  12  ENT  subjects. 


Snore 

Trialtime _ Navtime _ Navsnnre _ 

njtime  . 

Injsoore 

Disstime. — 

Dififirnre _ ^ 

Score 

1 

Trialtime 

-.965 

1 

Navtime 

-.587 

.469 

1 

Navscore 

.68 

-.573 

-.97 

1 

Initime 

-.531 

.508 

-.109 

.092 

1 

Injscore 

.684 

-.667 

.016 

.033 

-.957 

i 

Disstime 

-.784 

.897 

.189 

-.342 

.3 

-.476 

1 

Disscore 

.856 

-.918 

-.275 

.413 

-.352 

.524 

-.964 

1 

ESS  Perf. 

.504 

-.614 

-.245 

.227 

-.425 

.428 

-.526 

.471 

As  before,  prior  ESS  experience  is  predictive  of  overall  trial  time  (r  =  -.614),  and,  although  its 
relationship  to  each  of  the  subtask  measures  is  consistent  with  simulator  validity,  none  of  these 
correlations  are  statistically  significant. 

Novice  Model  Performance  Across  ENT  Groups 

Figure  5  shows  the  breakdown  of  first-time  novice  (abstract)  model  scores  for  this  group.  The  linear 
relationship  between  overall  novice  trial  score  and  year  of  residency  is  apparent  from  this  mapping. 
It  is  interesting  to  note  that  the  R5  residents  appear  to  have  higher  scores  on  the  novice  model  than 
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the  experienced  ENT  staff  subjects.  This  could  be  due  to  a  number  of  factors,  including  the  relatively 
high  number  of  procedures  performed  routinely  by  the  R5s. 


Figure  5.  Overall  scores  for  first  trial  on  novice  model  for  ENT  staff  and 
ENT  residents,  plotted  by  year  of  residency. 
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Figure  6.  Overall  trial  times  for  first  trial  on  novice  model  for  ENT  staff  and 
ENT  residents,  plotted  by  year  of  residency. 

Similarly,  it  can  be  seen  in  Figure  6  that  overall  trial  times  for  the  initial  novice  trials  fall  off  as 
expected  with  year  of  residency.  Both  of  these  findings  provide  strong  evidence  for  the  validity  of 
the  simulator  for  the  ESS  task,  even  when  the  model  is  abstract. 
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Subtask  Times  By  Year  of  Residency 

A  breakdown  of  trial  times  by  subtask  is  shown  for  the  range  of  ENT  subjects  in  Figures  7-9. 


Staff  Residents 


Figure  7.  Navigation  times  for  first  trial  on  novice  model  for  ENT  staff  and 
ENT  residents,  plotted  by  year  of  residency. 

While  resident  subtask  times  generally  decrease  with  year  of  residency,  dissection  times  indicate  a 
nonlinear  trend,  with  residents  in  the  middle  years  of  training  taking  longer  to  dissect  than  the  R5 
residents  (who  have  had  considerably  more  experience).  This  effect  may  perhaps  be  due  to  the 
degree  of  caution  observed  by  the  R3s  and  R4s  as  they  become  aware  of  the  procedural  risks  during 
dissection  but  do  not  yet  have  their  instrument  control  skills  well  honed. 


Staff  Residents 

Figure  8.  Injection  times  for  first  trial  on  novice  model  for  ENT  staff  and 
ENT  residents,  plotted  by  year  of  residency. 
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Figure  9.  Dissection  times  for  first  trial  on  novice  model  for  ENT  staff  and 
ENT  residents,  plotted  by  year  of  residency. 

These  subtask  times  for  the  staff  ENTs  suggest  that  injection  skill  is  well  learned  by  this  time,  while 
the  finding  of  relatively  more  variability  in  navigation  and  dissection  times  may  reflect  differences  in 
style,  with  some  experienced  surgeons  remaining  (appropriately)  more  cautious  and  perhaps  more 
attentive  to  the  potential  hazards  of  making  a  mistake. 

Subtask  Scores  By  Year  of  Residency 

A  similar  breakdown  for  subtask  scores  (shown  in  Figures  10-12)  mimics  the  trends  for  subtask  time. 


Figure  10.  Navigation  scores  for  first  trial  on  novice  model  for  ENT  staff  and 
ENT  residents,  plotted  by  year  of  residency. 
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Scores  on  each  subtask  generally  increase  with  year  of  residency,  with  the  exception  of  R3s  and  R4s 
achieving  slightly  lower  scores  for  dissection  than  their  junior  and  senior  colleagues.  Injection  scores 
for  R5s  are  comparable  with  staff  ENTs,  while  R5  navigation  and  dissection  scores  appear  to  be 
superior. 


Staff  Residents 


Figure  22.  Injection  scores  for  first  trial  on  novice  model  for  ENT  staff  and 
ENT  residents,  plotted  by  year  of  residency. 


Staff  Residents 

Figure  12.  Dissection  scores  for  first  trial  on  novice  model  for  ENT  staff  and  ENT  residents. 
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Novice  Model  Performance  Across  Subject  Groups 

Figure  13  shows  the  distribution  on  overall  scores  on  the  first  novice  trial  for  all  three  subject  groups, 
while  Figure  14  shows  the  same  for  overall  trial  times. 
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Figure  13.  Overall  scores  for  first  trial  on  novice  model  for  the  non-physician  group, 
the  non-ENT  physician  subjects  (representing  three  specialties),  and  the  ENT  group. 
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Trialtime 

Figure  14.  Overall  trial  times  for  first  trial  on  novice  model  for  the  non-physician  subject  group, 
the  non-ENT  physician  subjects  (representing  three  specialties),  and  the  ENT  group. 
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The  non-physician  group  is  clearly  less  skilled  than  the  ENT  group  (as  discussed  above),  while  the 
non-ENT  physicians  demonstrated  more  variable  performance.  Note  the  two  videoendoscopic 
surgeons  had  identical  overall  scores  on  their  novice  trials  (and  overall  times  that  were  essentially  the 
same,  as  well),  and  that  their  performance  was  comparable  to  the  average  ENT  subject.  The 
anesthesiologist  also  did  surprisingly  well,  due  perhaps  to  that  subject's  extensive  clinical  experience 
performing  image-guided  and  vital  sign-guided  procedures  which  require  new  psychomotor  skills. 

Initial  Intermediate  Model  Performance 

Figure  15  shows  the  distribution  of  trial  scores  on  the  initial  intermediate  trials  for  the  12  ENT 
subjects.  (Note  that  the  first  intermediate  trial  for  one  of  these  subject  was  terminated  after  the 
navigation  phase  and  is  consequently  not  included  here.) 

5f - - — - - — * — * — * - * — * — » — * — *■ — ‘ — *T 


Score 

Figure  15.  Frequency  distribution  of  overall  scores  on  initial  trial 
on  the  intermediate  model  by  subjects  in  the  ENT  group. 

Although  we  might  expect  a  positive  relationship  between  ESS  experience  and  novice  model 
performance,  we  would  expect  that  OR  experience  would  be  especially  predictive  of  performance 
when  the  sinus  anatomy  serves  as  the  model.  A  matrix  of  these  correlations  appears  in  Table  3 
(critical  r  =  .6319  for  p  <  .05,  df  8). 

In  contrast  to  the  initial  novice  scores,  these  results  indicate  that  the  best  predictor  of  initial 
intermediate  trial  time  (from  among  the  surgical  experience  measures)  is  subject  age  (r  =  -.475),  but 
this  correlation  is  not  statistically  significant.  For  trial  score,  the  findings  are  also  inconclusive,  with 
the  best  experiential  predictors  being  subject  age  (.581)  and  number  of  ESS  cases  performed  (r  =  .555). 
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Table  3.  Correlation  matrix  showing  relationships  among  ESS  experience  measures 
and  overall  performance  measures  on  the  intermediate  model  for  the  12  ENT  subjects. 


Score 

ESS  performed 

Yearsjraining 

Age 

Trialtime 


Score  FSS  perf  Years  tr  Age _ Trialtime 


1 

.555 

1 

.356 

.931 

1 

.581 

.667 

.68 

1 

-.795 

-.144 

-.014 

-.475 

1 

Note:  1  case  deleted  with  missing  values. 

In  general,  we  see  that  initial  performance  on  the  intermediate  model  is  indeed  positively  correlated 
with  prior  ESS  experience,  but  the  relationship  is  not  as  strong  as  for  initial  novice  model 
performance.  Table  4  attempts  to  shed  light  on  this  unexpected  finding  by  looking  more  closely  at 
experiential  correlates  of  performance  on  procedural  subtasks  for  the  initial  intermediate  trial  (critical 
r  =  .6319  for  p  <  .05,  df  8). 

Table  4.  Correlation  matrix  showing  relationships  among  ESS  experience  measures 
and  subtask  performance  measures  on  the  intermediate  model  for  the  12  ENT  subjects. 


ESS  perf... 
Years_tr... 
Age 

Navtime 

Navscore 

Injtime 

Injscore 

Disstime 

Disscore 


1 

.931 

1 

.667 

.68 

1 

-.569 

-.314 

-.386 

1 

.509 

.288 

.36 

-.971 

1 

-.869 

-.847 

-.624 

.435 

-.413 

1 

.859 

.798 

.525 

-.476 

.446 

-.967 

1 

.199 

.229 

-.428 

.191 

-.224 

-.275 

.362 

i 

-.059 

-.144 

.462 

-.354 

.381 

.186 

-.226 

-.967 

Note:  1  case  deleted  with  missing  values. 

It  appears  from  Table  4  that  prior  ESS  experience  is  indeed  a  strong  predictor  of  performance  on  the 
initial  intermediate  injection  task  (r  =  -.859  for  injection  time  and  .859  for  injection  score)  and  a 
slightly  weaker  predictor  of  performance  on  the  initial  intermediate  navigation  task  (r  =  -.569  for 
navigation  time  and  .509  for  navigation  score). 

Performance  on  the  initial  intermediate  dissection  task,  however,  is  actually  the  reverse  of  our 
predicted  effect:  dissection  times  are  slightly  longer  for  more  experienced  surgeons  on  their  first 
encounter  with  the  intermediate  model,  and  dissection  scores  are  slightly  lower. 

These  findings  are  deserving  of  further  study.  One  possible  explanation  might  be  that  there  is 
significant  negative  transfer  to  the  simulated  dissection  task  due  to  more  extensive  experience  with 
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the  real  instruments  and  real  anatomical  dissection.  Comments  from  experienced  ESS  surgeons 
regarding  the  difficulty  of  the  dissection  task  support  this  notion. 

Intermediate  Model  Performance  Across  ENT  Groups 

As  shown  in  Figures  16  and  17,  scores  on  the  initial  intermediate  trial  generally  improved  with  year 
of  residency.  Overall  trial  times,  on  the  other  hand,  appear  to  have  remained  relatively  constant,  but 
perhaps  less  variable,  over  residency  training  year. 


Figure  16.  Overall  scores  for  first  trial  on  intermediate  model  for  ENT  staff  and 
ENT  residents,  plotted  by  year  of  residency. 

While  intermediate  trial  times  might  be  expected  to  be  lower  than  those  for  the  residents,  their  real 
world  experience  with  the  procedure  may  actually  make  them  more  cautious,  as  noted  above.  The 
experienced  sinus  surgeon  knows,  for  example,  that  if  he  traumatizes  the  septum  or  middle  turbinate 
during  the  navigation  phase,  it  will  create  bleeding  that  will  affect  the  rest  of  the  surgical  procedure. 
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Figure  17.  Overall  times  for  first  trial  on  intermediate  model  for  ENT  staff  and  ENT  residents. 

Intermediate  Model  Performance  Across  Subject  Groups 

Overall  times  and  scores  for  initial  intermediate  model  trials  are  broken  down  by  specialty  in  Figures 
18  and  19.  As  expected,  the  ENT  group  performed  better  on  the  anatomical  model  than  did  the  non¬ 
physician  group. 
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Figure  18.  Overall  trial  times  for  first  trial  on  intermediate  model  for  the  non-physician  group , 
the  non-ENT  physician  subjects  (representing  three  specialties),  and  the  ENT  group. 
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Figure  19.  Overall  scores  for  first  trial  on  intermediate  model  for  the  non-physician  group, 
the  non-ENT  physician  subjects  (representing  three  specialties),  and  the  ENT  group. 


While  the  trial  time  difference  between  the  ENT  and  non-physician  groups  was  not  statistically 
significant,  the  trial  score  difference  was  significant  (as  summarized  in  the  table  below).  This  finding 
again  suggests  that  ESS  experience  is  predictive  of  simulator  performance,  thus  providing  evidence 
for  the  validity  of  the  system  for  ESS  training. 


Unpaired  t-Test  X 

DF:  l 

1 ;  specialty 

Jnpaired  t  Value: 

Y  i : score 

Prob.  (2-tail): 

1  12 

-3.533 

.0041 

Group:  Count: 

Mean:  Std.  Dev.:  Std.  Error: 

nonMD 

3 

53.667 

6.11 

3.528 

ENT 

11 

76.727 

10.631 

3.205 

Figures  18  and  19  also  show  that  the  non-ENT  physicians  in  this  sample  had  scores  that  were 
comparable  to  the  ENT  group.  (Note  that  the  radiologist  from  the  novice  trial  comparisons  above 
was  not  tested  on  the  intermediate  or  advanced  models.)  This  is  not  surprising,  given  that  these 
subjects  were  videoendoscopic  surgeons  who  typically  perform  more  endoscopic  procedures  on  a 
regular  basis  than  do  ENT  surgeons. 

General  videoendoscopic  procedures  may  also  be  more  challenging  from  a  psychomotor  perspective. 
In  addition  to  navigation,  injection  and  dissection,  their  procedures  may  require  retracting,  cautery, 
cutting,  suturing  and  true  dissection,  with  the  reflection  of  tissues  away  from  the  area  of  interest. 
Furthermore,  they  perform  these  task  in  a  more  bi-dexterous  fashion;  the  ENT  surgeon  uses  both 
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hands,  but  usually  one  at  a  time.  Finally,  the  videoendoscopic  surgeon’s  psychomotor  abilities  may 
be  challenged  more  during  their  procedures,  because  the  endoscope  and  instruments  are  usually  not 
coaxial. 

Initial  Advanced  Model  Performance 
Advanced  Model  Performance  Across  ENT  Groups 

Figures  20  and  21  show  the  overall  scores  and  trial  times  for  the  first  advanced  trial  by  subjects  in  the 
ENT  group.  Note  that  two  of  the  staff  subjects  did  not  attempt  the  advanced  model. 


Figure  20.  Overall  trial  scores  for  first  trial  on  advanced  model  for  ENT  staff  and 
ENT  residents,  plotted  by  year  of  residency. 

Surprisingly,  initial  performance  on  the  advanced  model  revealed  a  tendency  for  the  more  senior 
residents  to  perform  at  about  the  same  speed  as  the  staff  ENT  subjects  but  with  slightly  lower  overall 
scores.  This  may  reflect  the  ability  of  the  more  experienced  surgeons  to  perform  maneuvers  with  a 
higher  degree  of  accuracy. 
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Figure  21.  Overall  trial  times  for  first  trial  on  advanced  model 
for  ENT  staff  and  ENT  residents. 

While  not  a  strictly  linear  relationship  across  years,  the  R3  and  R4s  had  shorter  trial  times  on  their 
first  advanced  trial,  but  the  R4s  also  had  lower  scores  on  those  trials,  indicating  that  their  accuracy 
was  lower. 


Figure  22.  Injection  scores  for  first  trial  on  advanced  model  for  ENT  staff  and 
ENT  residents,  plotted  by  year  of  residency. 
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While  overall  time  and  score  for  staff  ENTs  did  not  appear  to  differ  systematically  from  the  resident 
times  and  scores  on  initial  advanced  model  trials,  their  performance  on  the  injection  subtask 
appeared  to  be  superior  to  the  resident  group  (as  shown  in  Figures  22  and  23).  Similar  differential 
performance  was  not  seen  for  the  navigation  and  dissection  subtasks. 


Figure  23.  Injection  times  for  first  trial  on  the  advanced  model  for  ENT  staff  and 
ENT  residents ,  plotted  by  year  of  residency. 

Although  navigation  and  dissection  may  vary  with  individual  care  and  priorities,  the  data  suggest 
that  injection  efficiency  may  be  a  hallmark  of  the  experienced  sinus  surgeon.  This  finding  should  be 
followed  up  and  verified  in  future  evaluation  studies. 

3.2.4  Asymptotic  Performance 

While  extensive  learning  curve  data  are  not  available,  repeated  measures  on  the  steady-state  system 
(version  1.2)  are  available  for  our  two  primary  evaluation  proctors  (CE,  an  ESS  surgeon,  and  CA,  an 
engineering  student). 

Given  that  these  two  subjects  had  each  tested  the  simulator  in  various  degrees  of  development 
approximately  30-40  times  over  a  6-8  month  period,  their  later  scores  may  provide  an  estimate  of 
asymptotic  performance  on  the  simulator. 

ENT  Steady-State  Performance 

Simulator  performance  scores  were  analyzed  for  the  last  11  completed  steady-state  trials  by  the 
experienced  ENT  proctor.  As  can  be  seen  in  Figure  24,  steady  state  trial  times  for  the  novice  model 
(mean  =  259  seconds,  s.d.  =  15.9)  were  lower  than  those  for  the  intermediate  and  advanced  models 
(combined  mean  =  512  seconds,  s.d.  =  57.1).  The  similarity  between  trials  times  on  the  intermediate 
and  advanced  models  may  suggest  that  the  two  tasks  are  comparable  for  the  highly  experienced 
subject  and,  further,  that  the  intermediate  model  training  aids  are  no  longer  necessary  for  this 
subject. 
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Figure  24.  Asymptotic  overall  trial  times  on  each  model  for  the  experienced  ENT  proctor. 

Times  are  plotted  in  order  of  trial  presentation. 

It  also  appears  from  this  figure  that  trial  times  for  each  model  are  consistently  lower  in  the  last  trial 
than  in  the  previous  trial,  suggesting  that  these  trials  times  may,  in  fact,  not  represent  steady  state 
performance.  An  examination  of  overall  scores  for  these  trials  (see  Figure  25)  reveals,  however,  that 
for  at  least  some  of  these  conditions  the  decrease  in  overall  trial  time  for  these  later  trials  came  at  the 
cost  of  trial  score.  It  should  be  noted  that,  since  this  subject  achieved  the  minimum  required  time  on 
all  subtask  trials,  these  trial  scores  precisely  reflect  a  decrement  in  performance  accuracy. 

Given  the  better  trial  score  for  the  first  samples  for  each  model  condition,  we  may  tentatively  treat 
the  trial  times  for  those  first  samples  as  more  representative  of  optimal  performance. 


Figure  25.  Asymptotic  overall  trial  scores  on  each  model  for  the  experienced  ENT  proctor. 
Scores  are  plotted  in  order  of  trial  presentation. 
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Figure  26.  Asymptotic  navigation  times  on  each  model  for  the  experienced  ENT  proctor. 

Times  are  plotted  in  order  of  trial  presentation. 

It  may  also  be  instructive  to  look  at  asymptotic  performance  for  the  three  subtasks  for  this  highly 
experienced  ENT  subject.  In  Figures  26-28  we  present  the  subtask  times  for  this  same  set  of  trials. 

As  can  be  seen  from  these  figures,  the  systematic  decrease  in  overall  trial  time  for  intermediate  and 
advanced  model  (shown  in  Figure  24)  appears  to  be  due  primarily  to  a  systematic  decrease  in 
completion  time  for  the  dissection  subtask,  while  navigation  and  injection  times  appear  to  vary  more 
randomly  across  these  temporal  samples. 


Figure  27.  Asymptotic  injection  times  on  each  model  for  the  experienced  ENT  proctor. 
Times  are  plotted  in  order  of  trial  presentation . 
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Figure  28.  Asymptotic  dissection  times  on  each  model  for  the  experienced  ENT  proctor. 
Times  are  plotted  in  order  of  trial  presentation. 


Observations 


Figure  29.  Asymptotic  navigation  scores  on  each  model  for  the  experienced  ENT  proctor. 
Scores  are  plotted  in  order  of  trial  presentation. 
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Figure  30.  Asymptotic  dissection  scores  on  each  model  for  the  experienced  ENT  proctor. 

Scores  are  plotted  in  order  of  trial  presentation. 

Similarly,  examination  of  Figures  29  and  30  suggests  that  the  overall  decrease  in  trial  score  for  the 
intermediate  and  advanced  models  for  this  experienced  subject  are  due  to  decreases  in  navigation 
score  for  the  advanced  model  condition  and  a  decrease  in  dissection  score  for  one  of  the  intermediate 
models  (intermediate  0  right).  Injection  performance  and  performance  on  the  navigation  and 
dissection  subtasks  for  the  other  models  appear  to  indeed  be  at  steady-state  asymptotic  levels.  (Note 
that  injection  subtask  scores  were  at  100  percent  for  each  of  these  11  trials,  and  are  consequently  not 
charted  here.) 

Collapsing  across  model  over  these  11  trials  for  the  expert  ENT  subject,  we  observe  the  Pearson 
product-moment  correlation  coefficients  ("r")  shown  in  Table  5  among  these  performance  variables 
(critical  r  =  .6021  for  p  <  .05,  df  9). 

Table  5.  Correlation  matrix  shoioing  relationships  among  subtask  and 
overall  performance  measures  across  the  final  11  trials  by  the  experienced  ENT  proctor. 


Trialscnre 

Navtima 

Navsonre_  _ 

Injtime 

nisstima 
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Trial  time 

Trialscore 
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Navtime 

.037 
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Navscore 

.856 

.119 

1 

Injtime 

-.718 

.384 

-.608 

1 

Disstime 

-.643 

.29 

-.387 

.642 

1 

Disscore 

.557 

-.136 

.049 

-.416 

-.593 

1 

Trial  time 

-.609 

.424 

-.357 

.689 

.987 

-.581 
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These  findings  suggest  that  (for  this  subject,  at  least)  the  best  asymptotic  predictor  of  overall  trial 
score  is  navigation  score  (r  =  .856),  and  that  the  best  predictor  of  overall  trial  time  is  dissection  time  (r 
=  .987),  as  suggested  above. 

Non-ENT  Proctor  Steady-state  Performance 

When  we  look  at  the  correlations  among  these  same  performance  variables  for  the  last  six  trials  of  the 
other  highly  experience  proctor  (CA),  we  see  the  same  result,  as  shown  in  Table  6  (critical  r  =  .8114 
for  p  <  .05,  df  4).  Again,  the  best  predictor  of  overall  trial  score  is  navigation  score  (r  =  .924),  and  the 
best  predictor  of  overall  trial  time  is  dissection  time  (r  =  .65 7),  although  the  latter  relationship  is  not 
statistically  significant. 

Indeed,  if  we  plot  the  trial  times  and  trial  scores  for  this  proctor  side-by-side  with  the  values  for  the 
expert  ENT  proctor,  we  see  that  they  are  remarkably  similar.  In  Figures  31  and  32  we  see  the  trial 
times  and  scores,  respectively,  for  the  non-ENT  proctor  on  the  left  side  of  each  graph  and  the  expert 
ENT  proctor  on  the  right  side  of  each  graph.  Again,  trial  times  on  the  novice  model  are  in  the  250- 
second  range,  while  trial  times  for  the  other  model  condition  are  in  the  550-second  range  for  the  non- 
ENT  proctor. 

Table  6.  Correlation  matrix  showing  relationships  among  subtask 
and  overall  performance  measures  across  the  final  six  trials  by  the  non-ENT  proctor. 


Trialscore 

Navtime 

Navscore. _ 
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Disstime 
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Trial  time 

Trialscore 
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Navtime 
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Injtime 
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1 
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Figure  31.  Comparison  of  asymptotic  trial  times  on  each  model  for  the  non-ENT  proctor 
and  the  experienced  ENT  proctor.  Trial  times  are  plotted  in  order  of  trial  presentation. 

These  findings  suggest  that  extensive  experience  with  the  simulator  may  afford  a  non-ENT  subject 
performance  values  which  are  comparable  with  an  experienced  ENT  proctor.  Assuming  the 
validation  findings  discussed  above,  this  bodes  well  for  the  training  effectiveness  of  the  simulator. 
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Figure  32.  Comparison  of  asymptotic  trial  scores  on  each  model  for  the  non-ENT  proctor 
and  the  experienced  ENT  proctor.  Scores  are  plotted  in  order  of  trial  presentation. 
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3.2.5  Post-Session  Questionnaire 

All  12  ENT  subjects  provided  us  with  post-session  questionnaire  data.  All  scale  responses  indicated 
here  are  for  10-point  scales  anchored  at  the  end  points. 

Proctor's  Instructions:  Most  subjects  found  the  proctor’s  instructions  "very  useful"  (6/12)  or 
"adequate"  (4/12).  The  one  area  singled  out  for  improvement  was  "endoscope  and  tool  handling." 

Simulator  Layout:  Simulator  layout  and  interaction  with  the  model  and  instruments  was  rated  as 
moderately  realistic  (mean  =  6.5  and  6.2,  respectively). 

Abstract  Model:  The  assessed  training  benefit  of  the  abstract  model  was  relatively  variable  (mean  = 
5.3,  s.d.  =  2.6),  perhaps  reflecting  the  variability  in  baseline  experience  of  the  subjects.  The  assessed 
training  value  of  each  of  the  five  "training  aids"  (hoops,  targets,  dissecting  spheres,  voice  feedback, 
and  heart  rate)  is  shown  in  Figure  33. 

As  can  be  seen  in  these  notched  box  plots  (which  plot  the  response  percentiles  around  the  median, 
with  the  center  of  the  notch  being  the  median  and  the  ends  of  the  box  representing  the  25th  and  75th 
percentile  score),  heart  rate  was  viewed  as  the  least  beneficial  training  aid  for  this  model. 


Figure  33.  Subject  ratings  of  the  training  value  of  each  training  aid 
in  the  abstract  ("novice")  model  (1  =  "none,”  10  =  "significant"). 

In  addition,  most  subjects  indicated  that  the  level  of  difficulty  of  the  abstract  model  should  remain 
the  same  (9  subjects  out  of  12)  or  increase  (3  out  of  12). 

Intermediate  Model:  Subjects  overwhelmingly  indicated  that  the  experience  with  the  novice  model 
prepared  them  for  Model  2  (mean  =  8.3,  s.d.  =  1.5).  Dissection  was  judged  as  more  difficult  in  Model 
2  than  in  Model  1  (median  =  7.1),  while  navigation  and  injection  were  seen  as  about  the  same  level  of 
difficulty  (median  =  5.5  and  5.0,  respectively). 
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Subject  ratings  of  the  benefits  of  the  Model  2  training  aids  are  shown  in  Figure  34.  As  is  evident  from 
the  non-overlapping  notches  in  the  box  plots,  the  heart  rate  cue  was  judged  to  be  significantly  less 
beneficial  than  the  hoops,  targets  or  voice  feedback.  Furthermore,  note  that  the  ratings  for  these 
training  aids  are  considerably  higher  for  the  intermediate  model  than  for  the  abstract  (novice)  model. 

One  possible  explanation  for  this  surprising  finding  is  that  in  the  novice  model  the  cues  were  useful 
for  training  psychomotor  skills  and  instrument  control,  while  in  the  intermediate  model  they  were 
also  useful  for  training  the  specific  surgical  procedure. 


Figure  34.  Subject  ratings  of  the  training  value  of  each  training  aid 
in  the  intermediate  model  (1  =  "none,"  10=  "significant"). 

Ratings  of  realism  of  the  anatomical  model  were  also  extremely  high,  with  a  median  across 
respondents  of  8.0  and  no  ratings  below  "6"  on  the  10-point  scale.  It  is  also  interesting  to  note  (in 
examining  the  correlations  among  survey  responses)  that  the  more  realistic  the  anatomical  model 
was  rated  by  these  subject,  the  easier  the  Model  2  subtasks  were  perceived  to  be  (r  =  -.503,  -.477,  and 
-.499,  for  navigation,  injection  and  dissection,  respectively). 

Advanced  Model:  Figure  35  presents  a  summary  of  the  distributions  of  responses  by  the  ENT 
subjects  to  all  questionnaire  items  regarding  the  advanced  model.  Responses  to  question  12 
(”model2  benefit")  indicate  that  the  vast  majority  of  these  subjects  felt  that  the  experience  with  Model 
2  was  highly  beneficial  for  performing  on  Model  3. 

In  rating  the  difficulty  of  Model  3  on  the  three  essential  subtasks  subjects  indicated  that  Model  3  was 
moderately  more  difficult  than  Model  2.  Finally,  the  level  of  procedural  realism  for  Model  3  was 
rated  quite  high,  with  a  median  of  8.0  on  the  10-point  scale  ranging  from  "far  from  reality"  to  "close 
approximation." 
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Figure  35.  Subject  ratings  of  the  training  benefit  of  Model  2  for  subsequent  performance  on  Model  3,  the 
difficult y  of  the  three  subtasks  (1  =  "Model  3  easier, "  10  =  "Model  3  more  difficult"),  and  the  fidelity  of  the 
simulation  in  the  advanced  model  (1  =  "far from  reality,"  10  =  "close  approximation"). 

Force  Feedback:  Seven  of  the  12  ENT  subjects  rated  the  use  of  the  haptic  subsystem.  When  asked 
whether  they  preferred  the  force  feedback  on  or  off,  4  preferred  it  "on,"  one  preferred  it  "off,"  and  one 
initially  preferred  it  "off"  but  later  preferred  it  "on."  When  asked  if  they  preferred  the  force  display 
on  the  forceps  or  on  the  endoscope,  6  indicated  a  preference  for  forces  on  the  forceps,  while  1 
indicated  a  preference  for  forces  on  the  endoscope.  (Note  that  they  only  experienced  the  forces  on 
the  forceps,  not  on  the  endoscope,  so  these  responses  were  partially  speculative.) 

Ratings  of  the  realism  of  the  force  feedback  were  moderately  low,  with  a  mean  of  5.1  and  a  standard 
deviation  of  1.7  on  the  10-point  scale  from  "far  from  reality"  to  "close  approximation,"  although  one  of 
the  7  subjects  rated  it  an  "8"  and  none  of  the  subjects  gave  it  a  realism  rating  below  "3." 

Overall  Evaluation:  Responses  to  items  regarding  the  overall  evaluation  of  the  simulator  are 
summarized  in  Figures  36  through  38.  Level  of  difficulty  of  the  simulator  for  the  three  subtasks  is 
assessed  in  Figure  36;  the  results  suggest  that  the  tasks  were  perceived  as  roughly  the  same  level  of 
difficulty  as  an  actual  procedure,  with  the  exception  of  the  dissection  task,  which  is  perceived  as 
somewhat  more  difficult  on  the  simulator. 
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Figure  36.  Subject  ratings  of  the  overall  difficulty  of  the  three  subtasks  compared  with  an  actual  procedure 

(1  =  "simulator  easier,"  10  =  "simulator  more  difficult"). 

In  assessing  the  training  value  of  each  model  for  themselves  (Figure  37),  there  appears  to  be  a  linear 
trend  from  the  novice  model  to  the  advanced  model,  with  all  seen  as  valuable  but  the  advanced 
model  especially  so. 


Figure  37.  Subject  ratings  of  the  overall  training  value  of  each  of  the  three  models 

(1  =  "none,"  10  =  "significant"). 
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In  assessing  the  adequacy  of  the  virtual  anatomical  model  for  developing  proficiency  at  the  common 
tasks  (Figure  38),  these  subjects  indicate  that  the  model  is  quite  adequate  for  training  navigation  and 
injection  skills,  but  may  be  less  than  adequate  for  training  dissection  skills. 


Figure  38.  Subject  ratings  of  the  adequacy  of  the  anatomical  model  for  developing  proficiency 
at  the  three  common  subtasks  (1  =  "inadequate,"  10  =  "more  than  adequate"). 

When  asked  to  indicate  what  levels  of  ENT  training  would  benefit  from  exposure  to  the  simulator, 
the  10  ENT  respondents  offered  the  following  distribution  of  responses,  suggesting  that  in  its  current 
implementation,  the  simulator  is  best  suited  for  junior  residents  (as  designed): 

med  student  =  2 

R1  =  6 

R2  =  6 

R3  =  4 

R4  =  2 

R5  =  2 

R6  =  0 

AH  =  4 

Finally,  would  further  realism  enhance  training  on  the  common  tasks?  All  10  respondents  indicated 
"yes,"  as  we  would  expect. 
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3.2.6  Optimal  Path  Analysis 

One  promising  technique  for  evaluating  user  performance  on  these  surgical  simulation  tasks  is  to 
look  at  the  paths  of  the  virtual  endoscope  and  virtual  tools  over  the  course  of  a  procedure.  A  sample 
of  such  paths  is  shown  in  Figure  39.  We  have  selected  a  sample  of  trials  to  illustrate  the  variability  in 
user  performance. 

The  colors  in  the  endoscope  path  renderings  (Panels  A,  B,  C,  and  D)  correspond  to  the  three  subtasks 
of  the  procedure  (navigation,  injection,  and  dissection)  during  a  trial  on  the  intermediate  model. 

Note  that  there  are  a  number  of  large  sweeps  outside  the  main  area  of  activity;  these  represent  the 
periodic  removal  of  the  'bloody"  scope  to  wipe  it  on  the  pad  on  the  forehead  of  the  mannequin. 

For  the  tool  paths  (E,  F,  G,  and  H)  the  colors  represent  the  dissection  of  each  sphere  in  the  novice 
model.  Since  the  tool  for  each  sphere  was  automatically  chosen  by  the  system  in  this  model  (to 
provide  subjects  with  a  systematic  exposure  to  the  instrument  set),  the  colors  in  these  paths  also 
represent  performance  with  the  various  tools. 

Tool  path  scores.  In  addition  to  drawing  the  paths  for  visual  analysis,  we  calculated  a  set  of 
objective  scores  based  on  the  tool  path  data.  The  total  time  taken  for  each  sphere  was  determined  by 
the  number  of  frames  for  that  segment.  These  measures  effectively  translate  into  overall  time,  since 
the  frame  rate  is  determined  primarily  by  position  of  the  Tenderer  in  the  anatomy,  which  was 
essentially  constant  across  subjects  for  each  (automatically  selected)  tool. 

A  second  measure  was  the  mean  deviation  of  the  tool  position  (in  each  frame)  from  the  centroid  of 
the  dissecting  sphere  for  that  tool.  A  third  (composite)  measure  was  then  calculated  for  each  sphere 
as: 

sphere  path  score  =  total  frames  *  mean  tool  deviation  from  centroid  / 1000 

Mean  frames/sphere,  mean  deviation  from  sphere  centroid,  and  mean  sphere  path  score  were  also 
calculated  for  each  subject’s  path,  yielding  an  overall  "path  score."  Note  that  for  each  of  these 
measures  (frames,  deviation,  and  path  score),  the  lower  the  score,  the  better  the  performance. 

First-order  correlations  among  these  new  path  measures  and  the  other  subject  performance  measures 
are  presented  in  Table  7  (critical  r  =  .4060  for  p  <.05,  df  22).  As  can  be  seen,  the  mean  frames/ sphere 
and  tool  path  scores  were  correlated  significantly  with  all  other  performance  measures,  and  most 
importantly,  dissection  score. 

In  addition,  the  mean  deviation/sphere  measure  correlated  significantly  with  dissection  score, 
overall  score,  trial  time  and  navigation  score.  Note  further  that  all  of  these  significant  correlations 
were  in  the  direction  expected.  These  results  may  provide  validation  evidence  for  the  system¬ 
generated  scoring  algorithm. 
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ENDOSCOPE  POSITION  DATA: 


TOOL  POSITION  DATA: 


E.  Staff  Otolaryngologist 

F,  Non-ENT  Subject 

Artifact 
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Tool  Position 

(dissection  spheres) 

G.  Jr.  Resident  Otolaryngologist 

H.  Sr.  Resident  Otolaryngologist 

9* 

> 

Figure  39.  Path  renderings  for  endoscope  and  tool  traversals. 
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Table  7.  Correlation  matrix  showing  relationships  between 
tool  path  measures  and  system-generated  subtask 
and  overall  performance  measures  for  the  initial  novice  trial. 
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Figure  40.  Linear  regression  for  novice  tool  path  score  by  system-generated  dissection  score, 
with  95%  confidence  limits  for  the  slope  of  the  regression  line,  broken  down  by  subject  group. 

The  strong  (inverse)  relationship  between  the  tool  path  scores  and  the  system-generated  dissection 
scores  can  be  seen  clearly  in  Figure  40.  It  is  interesting  to  note  that  the  relationship  appears  to  be 
even  stronger  for  the  ENT  group  alone. 
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Figure  41.  Scatterplot  showing  the  relationship  between  the  two  components 
of  the  tool  path  score  (mean  frames  per  tool ,  or  sphere,  and  mean  deviation 
from  the  sphere  centroids)  for  all  subjects  on  initial  novice  trial,  broken  down  by  subject  group. 

Figure  41  illustrates  the  relationship  between  the  two  components  of  the  tool  path  score  (essentially, 
speed  and  accuracy).  Points  closest  to  the  origin  on  both  axes  reflect  better  performance.  It  can  be 
seen  that  the  ENT  scores  cluster  much  closer  to  the  origin. 

In  examining  these  differences  further  using  students'  t,  we  find  that  the  non-ENT  subjects  did 
indeed  deviate  significantly  more  from  the  sphere  centroids  (ENT  mean  =  7.66,  s.d.  =  .807,  non-ENT 
mean  =  9.22,  s.d.  =  2.27,  p  =  .0422).  However,  their  time  scores  (ENT  mean  - 1883  frames/ sphere, 
s.d.  =  562.25,  non-ENT  mean  =  2381  frames/ sphere,  s.d.  =  1327.07,  p  =  .2162 )  and  overall  tool  path 
scores  (ENT  mean  =  15.06,  s.d.  =  6.49,  non-ENT  mean  =  23.24,  s.d.  =  15.33,  p  =  .1142),  while  worse, 
were  not  significantly  so. 

3.2.7  Proctor  Observations 

A  number  of  significant  insights  can  be  gleaned  from  subject's  open-ended  comments,  informal 
observations  by  the  proctor,  and  problems  experienced  during  trials.  Following  are  some 
observations  and  recommendations  derived  from  the  proctors’  experiences. 

Haptics 

Subjects  (most  notably  the  residents  and  ENT  staff)  had  difficulty  with  the  need  to  steady  their 
thumb  to  control  the  instrumented  forceps  during  injection  and  dissection.  This  stabilization  while 
opening  and  closing  the  forceps  was  required  to  keep  the  tip  of  the  virtual  tool  from  moving  away 
from  the  area  of  interaction.  This  need  for  stabilization  has  been  attributed  to  a  lack  of  realistic  haptic 
feedback. 

While  the  version  of  the  haptics  subsystem  tested  here  was  somewhat  rough.  Version  1.3  of  the 
software  (not  yet  formally  evaluated)  has  made  significant  improvements  to  the  haptics,  also 
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providing  a  much  more  realistic  representation  of  grasping,  tearing  and  injecting  the  virtual  tissue. 
Grasping  of  the  virtual  tissue  is  represented  by  a  "hold"  on  the  tip  of  the  instrumented  forceps,  after 
closing  the  jaw  on  the  tissue,  as  if  holding  onto  a  static  object.  Tearing  of  virtual  tissue  is  represented 
by  a  resistive  force  on  the  tip  of  the  instrumented  forceps  as  it  is  being  pulled  away  from  the  "hold" 
position,  with  a  final  "release"  after  a  predetermined  distance.  Injecting  is  represented  by  a  "pop" 
when  initially  passing  the  virtual  needle  through  the  tissue  and  a  "hold,"  keeping  the  instrumented 
forceps  static  in  the  X,Y,Z  position,  but  not  the  heading,  pitch  and  roll. 

In  addition  to  these  enhancements  for  stabilization,  the  suction  tool  is  "pulled"  toward  the  virtual 
tissue  based  on  its  proximity,  an  initial  "jolt"  is  placed  on  all  tools  when  initially  interacting  with  the 
virtual  tissue  and  the  feel  of  the  sickle  knife  simulates  cutting  paper  with  a  straight  razor.  These 
improvements  have  added  a  tremendous  amount  of  realism,  not  only  to  the  injection  and  dissection 
tasks,  but  also  to  the  realism  of  navigating  the  instrumented  forceps  through  the  sinus  cavity.  These 
improvements  will  be  implemented  and  tested  in  the  next  phase  of  the  simulator’s  development. 

Retraction  of  Instrumented  Forceps 

During  pilot  studies,  when  subjects  wanted  to  swap  tools,  they  were  required  to  retract  the 
instrumented  forceps  from  the  sinus  cavity  (retracting  to  just  posterior  to  the  columella  nasi)  before 
acquiring  the  new  virtual  instrument.  This  requirement  was  the  major  contributor  to  two  problems 
with  the  encoders  on  the  instrumented  forceps.  The  first  problem  took  place  during  the  pilot  studies; 
the  retraction  of  the  instrumented  forceps  caused  the  endoscope  shaft  to  collide  with  the  encoders 
measuring  the  heading  and  pitch  of  the  instrumented  forceps,  which  then  caused  either  fraying  or 
displacement  of  the  cables  on  the  sectors  of  the  encoders. 

The  second  problem  was  interaction  of  the  sectors  of  the  encoders  and  the  hard  palate  of  the 
mannequin  during  retraction.  This  interaction  forced  rotation  of  the  sectors,  introducing  an  offset  in 
the  initial  calibration  of  the  instrumented  forceps  and  therefore  an  offset  in  the  vector  of  the  virtual 
tool.  The  problem  was  resolved  by  applying  LocTite  to  the  sectors  and  eliminating  the  requirement 
that  the  subjects  retract  the  forceps  during  instrument  swapping.  Immersion  Corporation  is 
currently  upgrading  and  improving  the  design  of  the  instrumented  forceps  encoders  for  the  next 
phase  of  the  evaluation. 

Hummer  Speed 

The  typical  microdebrider  in  the  OR  operates  at  3000  RPM.  Experienced  surgeons  develop  a  certain 
rhythm  to  dissection,  and  expect  and  anticipate  a  certain  volume  of  tissue  to  be  removed  over  time. 

In  version  1.2  of  the  simulator,  the  dissection  speed  is  at  best  "quarter  speed."  More  experienced 
surgeons  perhaps  became  impatient  with  this  rate  of  dissection,  and  may  have  had  a  tendency  to 
move  on  to  another  area  of  the  anatomy  to  dissect,  to  the  detriment  of  their  dissection  scores.  It  also 
appeared  frustrating  to  have  areas  in  version  1.2  that  should  have  been  dissectable  but  would  not 
dissect. 

Ambiguity  in  the  Posterior  Ethmoid 

In  the  posterior  ethmoid,  the  model  takes  on  an  amorphous  look,  making  it  somewhat  difficult  to 
determine  what  to  dissect  next.  The  inexperienced  sinus  surgeon  would  simply  dissect  anything  that 
would  disappear.  For  the  experienced  surgeon,  who  knows  that  you  cannot  dissect  with  reckless 
abandon  for  fear  of  creating  a  major  complication,  dissection  naturally  slows  down  in  this  area  close 
to  the  carotid  artery,  optic  nerve  and  skullbase.  In  addition,  the  surgeon  normally  relies  heavily  on 
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haptic  cues  on  dissecting  posterior  ethmoid  cells.  The  absence  of  haptic  cues  in  an  amorphous 
environment  creates  an  anxious  situation  for  the  experienced  sinus  surgeon.  The  improved  haptic 
system  in  version  1.3  is  likely  have  significant  impact  on  dissection  performance  in  this  region. 

Grabbing  Bone  Fragments  with  Jawed  Tools 

Removal  of  two  bone  fragments  from  the  uncinate  process  and  three  bone  fragments  from  the  bulla 
ethmoidalis,  which  were  required  for  a  complete  score,  was  unrealistic,  due  to  the  difficulty  in 
grasping  with  the  jaw  of  the  virtual  tool;  the  fragments  could  only  be  grasped  at  their  center. 
Regardless  of  skill,  subjects  were  required  to  learn  how  to  grasp  the  bone  fragments  in  the  simulator 
and  required  instruction  by  the  proctor  on  how  to  do  so.  Many  of  the  staff  and  resident  ENT 
surgeons  initially  tried  to  grasp  the  bone  fragments  on  their  edge,  as  they  would  in  a  real  surgical 
environment,  but  without  success.  Version  1.3  has  corrected  this  problem  by  allowing  a  jawed 
virtual  tool  to  grasp  the  bone  fragments  anywhere  along  its  circumference  as  well  as  at  its  center. 

Improvement  of  the  Virtual  Suction  Tool 

During  testing,  the  suction  tool's  only  function  was  to  reduce  the  volume  of  the  blood  spheres  inside 
the  anatomy  after  dissection  of  part  of  the  tissue.  It  was  brought  to  our  attention  that  in  a  surgical 
environment  the  ENT  surgeon  would  use  the  real  suction  tool  not  only  to  remove  blood,  but  also  to 
remove  small  amounts  of  tissue  and  mucous  in  the  posterior  ethmoid  cells,  and  bone  fragments 
throughout.  Version  1.3  allows  a  small  amount  of  dissection  by  the  virtual  suction  tool,  as  well  as  the 
ability  to  grasp  and  slowly  remove  bone  fragments. 

Need  to  Break  Up  Model  1  into  Separate  Tasks  for  Non-Physician  Group 

During  pilot  testing  of  the  non-physician  group,  it  was  discovered  that  an  initial  verbal  summary  of 
the  task  to  be  performed,  without  a  visual  example  of  the  task,  was  overwhelming  to  the  untrained/ 
unfamiliar  subject.  The  need  for  the  subject  to  memorize  each  subtask  and  also  to  gain  an 
understanding  of  what  was  needed  to  complete  the  task  proved  to  be  too  demanding.  The  original 
presentation  of  the  material  consisted  of  a  videotaped  introduction  to  the  simulator  (described  below) 
followed  by  verbal  instructions  by  the  proctor,  describing  each  subtask  (navigation,  injection  and 
dissection)  in  order  and  in  totality. 

The  subject  was  next  introduced  to  the  endoscope  and  instrumented  forceps  and  instructed  in  their 
functions  for  each  task.  Then  the  subject  was  asked  if  there  was  anything  which  needed  further 
explanation.  We  found  that  the  proctoring  instructions  needed  to  be  repeated  multiple  times 
throughout  the  trial  to  reinforce  what  the  task  entailed.  By  trial  and  error,  it  was  finally  decided  to 
split  each  subtask  into  three  independent  tasks  for  their  initial  trial  on  the  simulator.  This 
breakdown  was  required  only  for  their  initial  trial. 

Ability  to  "Push  Through"  the  Virtual  Anatomy  During  Trials 

While  performing  the  three  tasks,  subjects  had  the  ability  to  push  through  the  virtual  anatomy  with 
the  shaft  of  both  the  virtual  endoscope  and  the  virtual  tool.  The  staff  ENTs  expressed  greatest 
concern  with  the  virtual  tool  passing  through  the  virtual  anatomical  structures  during  injection  and 
dissection.  A  potential,  but  expensive  and  cumbersome,  solution  to  this  problem  would  be  imple¬ 
mentation  of  haptics  with  6  degrees  of  freedom  on  the  instrumented  tool.  This  solution  would  be  too 
cumbersome  to  implement  in  die  current  design  of  the  mannequin  box,  but  should  be  considered  for 
future  designs. 
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The  unrealistic  ability  of  the  virtual  endoscope  to  pass  through  the  anatomy,  however,  was  welcomed 
by  the  majority  of  the  staff  ENTs.  When  the  virtual  scope  passes  through  the  anatomy,  the  image  on 
the  monitor  disappears  and  the  screen  turns  black  until  the  position  of  the  instrumented  endoscope 
is  maneuvered  back  inside  the  anatomical  model.  This  solution  was  seen  as  helpful  because  the 
residents  were  taught  to  concentrate  on  staying  within  the  anatomy  for  the  duration  of  the  procedure 
without  potentially  traumatizing  real  tissue  in  the  operating  room.  This  was  seen  as  one  of  the  many 
advantages  of  using  the  simulator  to  gain  the  hand-eye  coordination  necessary  to  maneuver  the 
endoscope  through  the  anatomy. 

Ability  to  see  Maxillary  Ostium  for  Antrostomy 

Although  the  subjects  were  told  to  perform  a  limited  ethmoidectomy,  the  antrostomy  of  the 
maxillary  ostium  was  removed  from  the  requirements  during  the  procedure  on  the  simulator 
because  of  the  inability  to  realistically  view  the  ostium.  After  dissection  of  the  uncinate  process  in  an 
actual  procedure,  the  maxillary  ostium  would  be  in  view  with  a  zero  or  30°  scope  and  the  antrostomy 
would  easily  be  conducted.  After  dissecting  the  uncinate  process  in  the  simulation,  however,  to  view 
the  ostium  required  a  70°  scope,  with  unrealistic  positioning  of  the  scope  inside  the  sinus  cavity.  To 
correct  this  problem.  Version  1.3  allows  dissection  of  the  uncinate  process  more  superior/laterally 
and  inferior/laterally  than  Version  1.2.  This  added  realism  will  be  tested  and  evaluated  in  the  next 
phase  of  the  evaluation  of  the  simulator. 

Instructional  Video 

An  instructional  video  was  made  to  introduce  the  subjects  to  the  simulator  to  aid  in  standardizing 
instructions  across  subjects.  The  video  was  designed  to  instruct  students  on  the  procedure,  the 
anatomical  structures  inside  the  sinus  cavity,  and  all  available  tools  at  their  disposal. 

The  content  structure  of  the  original  videotape  is: 

Endoscope  and  Forceps 

•  Introduction  to  the  instrumented  endoscope 

-  How  to  handle  it 

-  The  need  and  process  for  stabilization  using  both  hands 

-  Hints  on  guiding  the  scope,  by  use  of  angles,  through  the  anatomy 

-  How  to  eliminate  the  effects  of  your  natural  tremor 

-  Introduction  to  transferring  the  tool  across  the  columella  nasi 

-  Instructions  on  cleaning  the  scope  when  it  becomes  ’bloody" 

-  Hints  on  alignment  of  the  scope  within  the  plane  of  the  mannequin  to  keep 
track  of  your  heading  and  orientation 

-  The  scope’s  ability  to  rotate  the  image,  by  axially  rotating  the  shaft  of  the  scope 

-  Explanation  of  30°  and  70°  scopes  and  their  uses  to  view  around  comers 

•  Introduction  to  the  instrumented  forceps 

-  Limitations  of  the  subject’s  ability  to  fully  retract  the  forceps 

-  Rationale  for  passing  the  forceps  across  the  columella  nasi 

-  Their  rigid  connection  to  the  mechanics  of  the  haptics  inside  the 
mannequin 

-  How  to  swap  tools  in  Models  2  and  3 

-  Calling  out  the  desired  virtual  instrument 
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-  Retracting  the  forceps  to  just  posterior  to  the  columella  nasi  to  receive 
the  desired  virtual  instrument 

Body  Positioning 

•  Keeping  your  body  parallel  with  the  mannequin 

-  Turning  head  to  see  the  monitor  instead  of  turning  body 

Anatomy 

-  Introduction  to  major  anatomical  structures  on  physical  "pull  away"  model 

-  Nasal  passage,  septum  and  nasopharynx 

-  Introduction  to  sinus  cavity  anatomical  structures  on  physical  "pull  away" 
model 

-  Superior,  middle  and  inferior  turbinates,  uncinate  process  and  bulla 
ethmoidalis 

-  Description  of  anatomical  structures  which  will  be  interacted  with  and 
removed  during  procedure 

-  Definition  and  procedure  for  a  limited  ethmoidectomy 

-  Description  and  demonstration  of  sites  for  injection  during  procedure  in 
models  2  and  3 

-  Medial  middle  turbinate,  root  of  uncinate  process  and  the  lateral  nasal 
wall 

-  Description  and  "pull  away"  of  dissection  tasks  for  models  2  and  3 

-  Medialization  of  middle  turbinate  and  dissection  of  uncinate  process, 
bulla  ethmoidalis  and  widening  of  the  maxillary  ostium 

Procedures 

•  Playback  of  video  of  novice  model,  performed  by  Dr.  Edmond,  no  voice-over  to 

allow  the  proctor  to  discuss  the  trial  with  the  subject 

•  Examples  of  all  virtual  instruments  available  to  the  subject 

•  Playback  of  video  of  intermediate  model,  performed  by  Dr.  Edmond,  no  voice-over 

to  allow  the  proctor  to  discuss  the  trial  with  the  subject 

•  Playback  of  video  of  Model  3,  performed  by  Dr.  Edmond,  no  voice-over  to  allow  the 

proctor  to  discuss  the  trial  with  the  subject 

The  video  proved  to  be  too  advanced  for  the  non-physician  group,  presenting  an  excessive  amount 
of  information  for  the  untrained  or  unfamiliar  subject.  Pilot  subjects  from  this  group  who  were 
shown  the  video  reported  being  confused  as  to  what  was  required  of  them.  Specifically,  these 
subjects  did  not  have  enough  time  to  comprehend  both  the  simulator  interface  and  the  purpose  of 
the  procedure  being  simulated. 

Future  work  is  planned  in  this  area  to  create  group-specific  videos  for  each  subject  group. 

Shortening  the  duration  of  the  instructional  video  will  make  the  testing  protocol  more  efficient,  and 
automating  more  of  the  proctoring  instructions  will  permit  a  single  proctor  to  complete  all  necessary 
tasks  without  difficulty. 
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4  CONCLUSIONS  AND  RECOMMENDATIONS 

4.1  Procedural  Validity 

The  validity  of  the  simulator  for  the  ESS  domain  is  suggested  by  a  number  of  findings: 

•  ENT  subjects  performed  better  than  non-physician  subjects  on  both  the  novice  (abstract) 
and  intermediate  (anatomical  with  aids)  models 

•  Initial  performance  on  the  novice  model  was  correlated  with  residency  level  and  degree  of 
prior  ESS  experience 

•  Patterns  of  difficulty  for  asymptotic  performance  on  the  simulator  seem  to  match  the 
typical  pattern  of  subtask  difficulty  in  the  OR 

•  Subject  ratings  of  the  realism  of  the  virtual  anatomical  model  were  consistently  high  on  the 
post-session  questionnaire  and  in  open-ended  comments 

•  Post-training  questionnaire  responses  confirm  that  the  simulator  was  generally  perceived 
as  valid  and  useful  for  ESS  training  by  the  ENT  subjects 

4.2  Curriculum  Design 

The  ESS  simulator  takes  medical  simulation  several  major  steps  forward  in  its  evolution.  Aside  from 
its  technical  accomplishments,  the  integration  of  a  well  thought-out  curricular  framework  allows  it  to 
take  advantage  of  virtual  reality  without  sacrificing  the  benefits  of  more  traditional  computer-aided 
instruction. 

Model  1  introduces  the  student  into  an  abstract  environment,  allowing  the  student  to  gain  the 
required  hand-eye  coordination  with  the  endoscope  and  the  special  skills  needed  to  maneuver  the 
instrumented  forceps,  without  requiring  them  to  concentrate  on  anatomy. 

Model  2  introduces  the  student  to  the  anatomy,  but  still  utilizes  the  training  aids  from  Model  1.  This 
model  gives  the  student  the  help  of  hoops  for  the  initial  passes  through  the  anatomy,  targets  for 
injection  areas  and  labels  on  the  anatomical  structures  with  which  interaction  is  necessary.  The 
educational  advantages  of  simulation  can  best  be  achieved  with  a  model  of  this  kind. 

Model  3  introduces  the  student  to  a  more  realistic  environment.  There  are  no  longer  any  training 
aids  to  guide  the  student  through  the  procedure.  For  navigation  of  the  scope,  the  student  must  rely 
on  what  was  learned  when  navigating  through  the  hoops  in  Model  2.  For  injection,  the  student  must 
remember  where  injection  of  the  vasoconstrictor  is  useful.  For  dissection,  the  student  has  no  labels  to 
indicate  what  anatomy  to  interact  with,  and  so  must  rely  on  what  was  learned  in  Model  2  to  perform 
the  procedure. 

4.3  Future  Directions  for  Evaluation 

4.3.1  Scoring  Algorithm  Validation 

Considerable  attention  was  given  to  the  scoring  algorithm  currently  implemented  in  the  system,  in 
an  attempt  to  capture  the  primary  clinical  performance  factors.  Still,  it  is  somewhat  complex  and  ad 
hoc,  and  deserves  more  detailed  analysis  and  validation. 
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In  the  next  evaluation  phase  we  will  attempt  to  correlate  the  algorithmic  scores  with  independent 
measures  of  surgical  skill,  and  adjust  the  optimal  values  appropriately.  A  further  refinement  will  be 
to  normalize  the  algorithm  across  models  to  facilitate  comparisons  and  arrive  at  a  common  criterion 
for  advancement. 

4.3.2  Optimal  Instrument/Scope  Path  Analysis 

The  optimal  path  techniques  introduced  above  are  compelling  because  of  their  "face  validity"  but  will 
require  further  development. 

Tool  path  analysis.  Preliminary  pilot  data  indicate  that  these  tool  path  efficiency  measures  closely 
reflect  observers'  subjective  rankings  of  path  "goodness."  The  validity  of  these  measures  will  be 
studied  more  formally  in  the  next  evaluation  phase.  In  addition  to  calculating  overall  scope  and  tool 
path  scores,  we  will  look  more  closely  at  user  performance  with  the  individual  tools  in  the  virtual 
instrument  set  and  at  the  effects  of  anatomical  position  on  performance  with  each  tool. 

Determination  of  the  clinically  acceptable  range  of  path  scores  is  desirable  for  several  reasons, 
including  the  development  of  training  protocols  that  reinforce  appropriate  skills.  The  range  of  path 
scores  is  bounded  on  the  low  end  by  the  radius  of  the  dissecting  spheres  and  the  minimal  time  to 
dissect  (which  is  not  known,  but  can  be  approximated  by  the  asymptotic  performance  of  an 
experienced  ENT  surgeon. 

The  constraints  on  the  high  end  are  not  as  clear  (at  least  for  the  time  component),  and  should 
perhaps  be  determined  clinically.  While  theoretically  a  surgeon  could  take  infinitely  long  to  perform 
a  procedure,  there  are  several  practical  and  clinical  constraints,  including  the  increased  risk  to  the 
patient  during  protracted  procedures. 

Scope  path  analysis.  We  have  not  yet  attempted  a  quantitative  analysis  of  the  endoscope  path  data. 
In  the  next  evaluation  phase  we  will  experiment  with  various  methods  of  determining  mean 
deviation  from  the  optimal  scope  path.  Initially  the  most  direct  path  between  the  training  aids 
(hoops,  targets  and  dissecting  spheres)  may  serve  as  an  optimal  path,  but  this  approach  does  not 
take  into  account  the  surgeon's  knowledge  and  skill  in  navigating  around  critical  anatomical 
structures. 

A  more  appropriate  approach  might  be  to  calculate  the  optimal  path  using  a  robust  lowess 
regression  technique  on  the  datasets  from  a  group  of  experienced  ENT  surgeons,  and  then  determine 
the  mean  deviation  from  that  optimal  path  for  each  subject. 

4.3.3  Evaluation  of  Improved  Haptics 

The  value  of  the  haptic  feedback  subsystem  to  the  simulator  was  assessed  only  subjectively  in  the 
current  phase.  In  general,  it  was  perceived  to  be  only  moderately  useful,  and  we  suspect  that  the 
relatively  poor  performance  on  the  dissection  task,  in  particular,  may  have  been  due  largely  to 
inadequate  feedback  about  the  tissue  forces. 

As  described  above,  preliminary  evaluation  of  Version  1.3  suggests  that  the  haptic  subsystem  is 
greatly  improved.  The  improved  system  will  allow  us  to  evaluate  more  formally  its  impact  on 
training  trial  performance  and  its  contribution  to  OR  performance.  In  addition,  we  will  investigate 
more  formally  the  relative  merits  of  force  feedback  on  the  endoscope  versus  the  instrument. 
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4.3.4  Transfer  of  Training 

Transfer  of  training  to  the  real  operative  environment  is  the  primary  objective  of  this  procedural 
simulator,  and  will  be  the  primary  focus  of  the  next  phase  of  our  research.  In  preparation  for  that 
phase,  we  have  collected  initial  OR  videotape  data  from  several  first-time  ESS  procedures  by 
residents  for  whom  we  have  previous  simulator  performance  data.  These  tapes  will  be  formally 
analyzed  for  subtask  proficiency,  as  rated  by  their  attending  staff  instructors,  to  derive  a  set  of 
systematic  measures  of  performance  on  the  target  task.  These  will  then  be  correlated  with  prior 
simulator  experience  for  current  and  entering  residents  to  determine  the  training  effectiveness  of  the 
simulator  and  to  provide  guidance  for  its  further  development. 

4.4  Model  Enhancements 

As  noted  earlier,  the  development  team  elected  to  focus  the  first  iteration  of  the  ESS  simulator 
(appropriately)  on  the  needs  of  junior  residents.  Several  enhancements  to  the  library  of  available 
anatomical  models  are  currently  being  integrated  which  are  intended  to  extend  the  utility  of  the 
system  to  senior  residents  and  more  experienced  staff  physicians. 

Appropriate  enhancement  implementation  will  require  iterative  formative  evaluation,  as  before,  and 
the  training  utility  of  each  enhancement  will  need  to  be  assessed.  Transfer  of  training  may  need  to  be 
assessed  indirectly  in  some  cases,  since  real-world  incidence  of  certain  conditions  of  interest  may  be 
infrequent  under  normal  conditions. 

4.5  Criterion  Performance  Standards 

One  longer-term  objective  of  this  evaluation  activity  will  be  to  determine  the  clinically  acceptable 
range  of  user  performance  scores  on  the  simulator,  including  those  generated  by  the  basic  scoring 
algorithm  and  those  calculated  from  the  optimal  path  data.  Further,  more  detailed  analysis  of  these 
scores  (e.g.,  time  vs  accuracy  for  specific  tools)  will  also  provide  us  with  useful  information  for 
customizing  training  protocols  for  individual  residents. 

Ideally,  subject  performance  on  the  simulator  will  be  reliably  predictive  of  OR  performance. 
Achievement  of  this  correspondence  will  enable  us  to  better  establish  performance  criteria  for 
advancement  through  the  protocol.  In  a  sense,  the  scoring  algorithm  represents  a  theoretical  model 
of  surgical  proficiency;  additional  validation  studies  of  the  simulator  will  help  to  correct  and  refine 
those  theoretical  models. 

In  addition  to  the  learning  curve  for  simulator  performance  over  time,  the  distribution  of  trials  over 
time  is  also  of  interest.  For  our  "steady-state"  proctors,  for  example,  the  temporal  distribution  of 
training  trials  was  highly  variable,  ranging  from  several  trials  per  day  to  one  trial  every  2-3  weeks. 
Routinely  collecting  repeated  measurements  on  resident  subjects  will  enable  us  to  explore  the 
optimal  number  and  timing  of  training  trials  in  the  residents'  curriculum.  Undoubtedly  some  degree 
of  trial  spacing  (as  opposed  to  massed  practice)  will  prove  most  effective,  as  has  been  shown  in  many 
other  domains;  of  interest  is  just  what  that  spacing  strategy  should  be  for  ESS  procedure  training. 

Ultimately,  we  may  also  be  able  to  establish  an  equivalence  between  time  on  the  simulator  and  time 
in  the  OR.  Such  a  correspondence  would  be  useful  for  residency  curriculum  development,  and 
perhaps  eventually  for  professional  credentialling.  Such  a  system  would  be  of  significant  benefit  for 
patients,  and  could  greatly  extend  accessibility  to  procedural  training  opportunities. 
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5  SUMMARY 

This  study  presents  the  results  of  our  systematic  evaluation  of  a  high-end  virtual  reality  simulator 
aimed  at  training  a  set  of  skills  essential  to  endoscopic  sinus  surgery.  Our  findings  suggest  that  the 
simulator  represents  a  valid  and  useful  implementation  of  the  target  ESS  tasks.  In  addition,  the 
thoughtful  integration  of  an  organized  curriculum  perspective  makes  this  system  uniquely  valuable 
among  emerging  medical  simulation  systems. 

This  study  also  suggests  a  framework  for  incorporating  systematic  evaluation  into  the  process  of 
developing  procedural  training  simulators  in  the  medical  domain.  Incorporating  both  formative  and 
summative  aspects  of  evaluation  has  greatly  enhanced  the  development  process  and  assures  the 
continuing  evolution  of  a  usable  and  effective  system. 
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reply  to 

ATTENTION  OF 


DEPARTMENT  OF  THE  ARMY 

HEADQUARTERS,  MADIGAN  ARMY  MEDICAL  CENTER 
TACOMA,  WASHINGTON  98431-5000 


18  October  1995 


As  you  recall,  Madigan  AMC  has  been  funded  to  develop  a  prototype 
surgical  simulator  for  endoscopic  paranasal  sinus  surgery.  The 
focus  of  the  project  is  to  explore  ways  of  using  advanced 
technology  in  the  form  of  a  surgical  simulator,  to  assist 
providers  with  acquiring  the  skills  necessary  to  perform  safe  and 
effective  endoscopic  sinus  surgery.  We  are  exploring  the  full 
range  of  training,  from  novice  to  expert. 

We  are  asking  a  select  group  of  physician  for  input  to  help  quide 
the  clinical  team  involved  with  the  development  of  the  simulator. 
Your  input  will  be  used  to  help  focus  our  efforts. 

As  you  know,  the  technology  is  advancing  rapidly.  Our  goal  is  to 
develop  a  visually  realistic,  highly  interactive,  patient 
specific  surgical  simulator  for  training.  However,  a  single 
solution  is  not  the  answer.  For  example,  what  is  important  for 
the  novice  is  not  necessarily  important  for  the  expert.  You  can 
assist  us  by  defining  what  is  important  at  each  level.  We  will 
analyze  this  information  and  use  it  to  develop  the  first 
iteration  of  the  simulator.  It  is  our  belief  that  in  the  future 
this  technology  will  contribute  significantly  to  the  develop  of 
future  Otolaryngologist. 

Sly,  if  you  could  distribute  this  questionnaire  to  Dr's  Hayes, 
Liening,  Teller  and  Malis,  I  would  appreciate  it.  Please  have 
them  return  it  as  soon  as  possible.  If  you  could  batch  them  and 
send  it  back  that  would  work  best. 


Director  of  Residency  Research 
Madigan  Army  Medical  Center 
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At  each  level  of  training,  the  needs  of  the  surgeon  differs. 
Please  score  from  1  (critical  need)  to  7  (not  important)  for  each 
level  of  training. 


NOVICE 

Med  Student- 

PGY3 


INTERMEDIATE 

PGY3-PGY4 


EXPERT 
PGY4 -BEYOND 


Visual  Realism 


Spatial 
Awareness 
(anatomic  3-D 
orientation 


Haptic 

Interaction 

(touch- 

feedback) 


Patient 

Specific  model 


P  sychomot or 
Development 


Interactivity 
(response  of 
application  to 
user) 


Computer-aided 

dissection 

(real-time 

localization 

during 

surgery) 


Demonstration 
of  standard 
surgical 
procedures 


Demonstration 

of 

complications 


Advanced 

surgical 

techniques 


Sinus 

pathophys .  / 

medical 

management 
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Please  rank  order  from  1-11  for  each  level  of  training  (novice¬ 
expert)  . 


NOVICE 

Med  Student  - 

PGY3 


INTERMEDIATE 

PGY3-PGY4 


Visual  Realism 


Spatial 
Awareness 
(anatomic  3-D 
orientation 


Haptic 

Interaction 

(touch- 

feedback) 


Patient 

Specific  model 


Psychomotor 

Development 

(hand-eye 

coordination) 


Interactivity 
(response  of 
application  to 
user) 


Computer-aided 

dissection 

(real-time 

localization 

during 

surgery) 


Demonstration 
of  standard 
surgical 
procedures 


Demonstration 

of 

complications 


Advanced 

surgical 

techniques 
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Sinus 

pathophys .  / 

medical 

management 
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In  paragraph  form,  please  describe  your  thoughts  as  they  pertain 
to  surgically  simulated  endoscopic  paranasal  sinus  surgery. 


1.  What  is  the  importance  of  visual  realism,  when  trying  to 
graphically  represent  anatomy  and  surgical  procedures  at  all 
levels  of  training? 


2.  How  important  is  anatomic  spatial  awareness  at  all  levels  of 
training? 


3.  While  simulating  the  operative  procedure,  how  important  is 
forced  feedback?  (  computer  generated  forces  that  mimic  the 
forces  encountered  in  an  actual  procedure) . 


4.  How  important  is  a  patient  specific  model? 
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5.  How  important  would  it  be  to  demonstrate  paranasal  sinus 
pathophysiology,  medical  management,  advanced  applications, 
surgical  complications  in  a  simulated  environment  or  as  computer 
aided  instruction? 


6.  Is  there  any  benefit  in  using  the  simulator  to  assist  the 
user  with  the  development  of  his/her  psychomotor  skills?  (ie. 
handling  the  rigid  scope  while  dissecting) . 


7 .  What  areas  should  we  address  that  are  not  included  on  this 
questionnaire? 
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TRIAL  OPERATIONS 


Name:  Subject#: 

Trial  Type: 

Injection 

|  Dissection  1 

End 

ESS 

1  Mid. 

End 

Training  Aids 

X 

Labels 

Targets 

Hazards 

Markers 

Scopes 

0° 

30° 

70° 

CT  View  | 

Tool 

X 

None 

, 

Complications  j 

Heart  Distress  | 

i 

i 

— 

Patient  Options  j 

X 

Bleeders 

X 

Audio  *] 

X 

Cues 

X 

Heart  Monitor 

Tool  Options  ™] 

|  X  |  Blood  E.  Scope 

_ 

Ret.  Req.  Chee 

Surgical  Instruments  | 

Freer 

Needle 

Bent  Needle 

Sickle  Knife 

Microdebrider 

Suction 

■ 

■ 

■ 

Left-biting 

Forceps 

■ 

■1 

Right-biting 

Forceps 

X 

NULL 

NOTES: 
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ENDOSCOPIC  SINUS  SURGERY  SIMULATOR 
Pre-training  Questionnaire 

S# _  Date  _____ 

Location _  Proctor  _ 

M _  F _ _  Age _  Left-Handed _  Right-handed _ 

Occupation:  MD _  Specialty: _ 

Type  of  Practice  (Academic/ research/community-based,  etc): 


Other _ _ _ 

Any  factors  that  might  affect  your  performance  today  (lack  6f  sleep,  medication,  etc): 

Videogame  experience:  _ none _ once _ occasional _ often 

Virtual  Reality  experience:  _ none _ once _ occasional _ often 

Simulator  experience:  none _ once _ occasional _ often 

Specify: _ __ 


If  non-oto  surgical  specialty: 

#  Years  of  surgical  training:  _ _  #  Years  of  surgical  practice: _ 

Endoscopic  procedures  performed:  _ 0  _ 1-5  _ 6-20  _ 20-100  _ >100 

Endoscopic  procedures  observed:  _ 0  _ 1-5  _ 6-20  _  20-100  _ >100 

If  otolaryngologist: 

#  Years  of  oto  specialty  training  : _  #  Years  of  oto  specialty  practice  : _ 

ESS  procedures  (sides)  performed:  0  1-5  6-20  20-100  >100 

ESS  procedures  (sides)  observed:  0  1-5  6-20  20-100  >100 

ESS  teaching  experience  (students):  _ 0  _ 1-5  _ 6-20  _ 20-100  _ >100 

How  did  you  learn  endoscopic  surgery  (courses,  cadaver  dissection,  etc)? 
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ENDOSCOPIC  SINUS  SURGERY  SIMULATOR 
ENT  Post-training  Questionnaire 

S# _  Date _ 

Location _  Proctor _ 


Simulator  Preparation 

1.  How  did  you  find  the  videotape  and  the  proctors  instructions? 

insufficient  adequate  very  useful 

2.  If  you  answered  "insufficient",  what  areas  of  instruction  do  you  feel 

could  benefit  from  more  detailed  emphasis? 


introduction 

endoscope  and  tool  handling 

body  positioning 

anatomy 

abstract  model 

model  with  aids 

model  without  aids 

OTHER 

OTHER  COMMENTS 

Simulator  Layout 

3.  From  your  operative  sinus  experience,  how  closely  does  the  simulator 

layout  resemble  your  operative  set-up? 

Not  at  all  Identical 

123456789  10 

4.  How  closely  does  your  interaction  with  the  physical  model  (mannequin) 

endoscope  and  forceps  resemble  an  actual  sinus  procedure? 

Not  at  all  Indistinguishable 

12345678910 


OTHER  COMMENTS. 


Abstract  (Novice)  model  -  The  intent  of  the  abstract  model  is  to  provide  a 
simple  environment  for  the  user  to  focus  completely  on  skills  enhancement 
(navigation,  injection  &  dissection).  The  following  questions  will  focus  on 
the  training  benefit  of  the  abstract  model. 

5.  At  your  present  endoscopic  skill  level,  what  training  benefit  did  the 
abstract  model  provide  you? 

None  Significant 

123456789  10 


160 


Appendix  I 


DAMD1 7-95-2-5023 
Final  Report 


6.  What  training  value  did  you  find  in  the  following  training  aids? 

None  Significant 


hoops 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

targets 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

dissecting  spheres 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

voice  feedback 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

heart  rate* 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

7.  Should  the  level  of  difficulty  be  changed  in  the  abstract  model? 

increased  no  change  decreased 

OTHER  COMMENTS _ 


Sinus  model  with  training  aids  present  (Model  2) 

8.  Did  your  experience  in  the  Novice  Model  prepare  you  for  Model  2? 

Not  at  all  Significantly 

123456  789  10 

9.  In  comparison  to  the  Novice  Model,  did  you  find  the  tasks  easier  or  more 

difficult  in  Model  2? 


Model  2 

Model  2 

Easier 

Same 

More  Difficult 

navigation 

1 

2  3  4  5  6  7  8 

9 

10 

injection 

1 

2  3  4  5  6  7  8 

9 

10 

dissection 

1 

2  3  4  5  6  7  8 

9 

10 

Were  the  training  aids  of  benefit  in  Model  2? 

None 

Significant 

hoops 

1 

2  3  4  5  6  7  8 

9 

10 

targets 

1 

2  3  4  5  6  7  8 

9 

10 

voice  feedback 

1 

2  3  4  5  6  7  8 

9 

10 

heart  rate 

1 

2  3  4  5  6  7  8 

9 

10 
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11.  Did  the  virtual  sinus  anatomy  geometrically  and  anatomically 
approximate  a  representative  endoscopic  sinus  surgery? 

Far  from  reality  Close  approximation 

123456789  10 


OTHER  COMMENTS. 


Sinus  model  without  training  aids  present  (Model  3) 

12.  Did  your  experience  in  Model  2  prepare  you  for  Model  3? 

Not  at  all  Significantly 

12345  6  789  10 

13.  In  comparison  to  the  Model  2,  did  you  find  the  tasks  easier  or  more 

difficult  in  Model  3? 

Model  3  Model  3 

Easier  Same  More  Difficult 


navigation 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

injection 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

dissection 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

14.  In  Model  3,  how  closely  does  the  simulation  replicate  an  actual  sinus 
procedure? 

Far  from  reality  Close  approximation 

123456789  10 


OTHER  COMMENTS 


Force  Feedback 

15.  Did  you  prefer  the  force  feedback  on  the  forceps?  On  Off 


16.  If  you  had  a  choice  between  having  forces  on  the  forceps  or  endoscope, 

which  would  you  prefer?  Forceps  Endoscope 

17.  How  closely  did  the  force  feedback  mimic  an  actual  procedure? 

Far  from  reality  Close  approximation 

123456789  10 


OTHER  COMMENTS. 
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Overall  Evaluation 

18.  How  does  the  level  of  difficulty  in  the  simulator  compare  with  an  actual 
procedure  for  the  three  tasks? 

Simulator  Simulator 

Easier  Same  More  Difficult 


navigation 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

injection 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

dissection 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

19.  At  your  present  level  of  experience,  what  training  value  (for  yourself)  did 
you  find  in  each  of  the  models? 

None  Significant 


Model  1 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Model  2 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Model  3 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

20.  Is  the  present  level  of  anatomic  representation  in  Model  2  and  3  adequate 
for  developing  proficiency  at  the  common  tasks? 

Inadequate  More  than  adequate 


navigation 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

injection 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

dissection 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

21.  What  levels  of  ENT  training  would  benefit  from  exposure  to  the 
simulator?  (circle  as  many  as  indicated) 

medical  student  R1  R2  R3  R4  R5  R6  ALL 

22.  In  your  opinion,  would  further  "realism"  enhance  training  in  the 
common  tasks  (navigation,  injection,  dissection)?  Yes  No 

23.  What  aspects  of  the  simulator  are  the  most  effective? _ 


24.  What  improvements  would  you  most  like  to  see?. 
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OTHER  COMMENTS. 
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Appendix  J  -  Questionnaire  Comments 


APPENDIX  J 
SUMMARY  OF 

QUESTIONNAIRE  COMMENTS 
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SUBJECT  COMMENTS 

The  following  comments,  organized  by  topic,  were  collected  from  the  Post  Session  Questionnaire 
and  notes  taken  by  the  proctors  during  data  collection  sessions.  Not  all  subjects  commented,  and 
redundant  comments  were  omitted  from  these  lists. 

WHAT  ASPECTS  OF  THE  SIMULATOR  ARE  MOST  EFFECTIVE  ? 

"Practice  in  moving  through  the  anatomy  with  similar  tools" 

"Instrument  movements  (jaw  movement)" 

"Working  with  instruments  in  tight  quarters" 

"The  image  of  the  anatomy  and  the  rings  to  show  you  the  way" 

"Learning  non-in tuitive  (i.e.  opposite)  movements  mechanical  vs  visual" 

"Organization  of  tasks  (step  to  step),  hands-on  instrumentation,  anatomy" 

"Anatomy,  hand  coordination" 

"Becoming  familiar  with  endoscopic  navigation  and  anatomy" 

"Injection/  dissection" 

"Liked  the  fact  that  the  simulator  was  not  'completely  realistic'" 

"Navigation/ instrument  manipulation,  simulation  of  techniques" 

"Using  training  aids,  then  stepping  up  to  same  scenario  without  training  aids" 

WHAT  IMPROVEMENTS  WOULD  YOU  MOST  LIKE  TO  SEE  ? 

"Better  3D  realism" 

"Remove  rotational  bias  of  endoscope" 

"Haptics  on  the  scope" 

"Tool  encoder  and  endoscope  shaft  interaction  needs  to  be  eliminated" 

"Less  need  to  stabilize  thumb  on  the  forceps  tool" 

"Improved  haptics,  improved  danger  areas  —  brain,  eye,  optic  nerve,  carotid  artery. 

Improved  posterior/ anterior  ethmoid  cells,  add  thickened  mucosa,  pus,  mucus" 

"Blood  droplets  should  disappear  instantly  and  not  take  too  long" 

"Audio/Visual  warnings  for  hazards  (Le.  arteries,  nerves,  lamina...)" 

"Resistance  to  "drift"  in  tool" 

"Want  an  elbow  stand  for  stabilization" 

"Place  and  endoscope  'sleeve'  on  the  scope  to  allow  better  grip" 

"Allow  suction  tool  to  break  open  ethmoid  air  cells" 

"Better  depth  perception  and  better  force  feedback  barrier  to  instrument  drift" 

"The  30  degree  scope  is  not  right.  It  still  resembles  a  zero  degree  scope" 

"The  view  when  rotating  the  30  degree  scope  is  unrealistic" 

"Ability  to  manipulate  mannequin  to  simulate  operative  positioning" 

"Simulation  of  complications  (entering  orbit,  penetrating  skullbase,  etc)" 
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"Surgical  setup  (patient  position,  endoscope  sleeve,  etc)  as  realistic  as  possible" 
"Forceps  less  sensitive /more  realistic" 

DID  YOU  FIND  THE  SIMULATOR  LAYOUT  REALISTIC,  INTUITIVE,  CUMBERSOME, 
UNREALISTIC  ? 

"Place  more  consideration  on  instrument  used  and  instrument  handling  rather  than 
reality  of  graphics" 

"Moderately  realistic" 

"Educational,  not  photorealistic" 

"Occasionally  had  difficulty  with  both  scope  and  forceps  positioning  in  same  location" 

COMMENTS  ON  MODEL  1  (Novice) 

"Did  not  notice  heart  rate  much  of  the  time" 

COMMENTS  on  MODEL  2  (Intermediate) 

"Training  aids  make  you  concentrate  on  the  aids  and  not  the  anatomy,  a  "distraction"; 
anteriorly,  they're  great,  posteriorly,  needs  work" 

COMMENTS  on  MODEL  3  (Advanced) 

"The  only  problem  is  with  the  forceps/  bone  chips.  Simulator  too  specific  where  you 
have  to  grab  bone." 

COMMENTS  on  FORCE  FEEDBACK 

"Did  not  really  appreciate  the  force  feedback" 

"Like  the  haptics  better  on  than  off’ 

"Want  the  haptics  to  ’tear’  the  tissue,  want  the  ability  to  feel  the  biting  of  the  tools" 
"It  was  hard  to  separate  Haptics  from  intrinsic  weight  of  tool  (inertia)" 

"Force  feedback  on  stem  of  instruments  and  on  camera" 

OTHER  COMMENTS 

"I  really  enjoyed  the  bloody  scope" 

"Helps  tremendously  with  hand  /eye  coordination,  anatomy,  etc" 

"Track  respiratory  rate  and  ECG  during  trials  to  determine,  stress  level  and  ability  to 
perform" 

"Overall  somewhat  difficult  and  frustrating  - 1  felt  I  had  to  unlearn  some  skills 
developed  in  real  surgery" 

"Impressive  technology" 

"Create  a  procedure  which  guides  the  residents  from  point  A  to  Z  through  the 
procedure.  Noticed  that  the  residents  tend  to  pause  during  the  actual 
procedures  because  they  have  no  plan'  to  follow" 

"A  good  tool.  Would  be  worthwhile  training  aid,  especially  for  those  learning  sinus 
surgical  techniques" 
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